We are constantly working on additional features for our users to ensure delivery of high quality service, thus we have implemented an add-on to Auto Rate Tasks feature.
This added feature in auto rate tasks is very useful to campaigns resulting to have random answers such as Evaluation of a certain subject, Annotating given conditions, Assessment of a certain product, etc. Rating proofs for such campaigns is a very tough job but with this add-on auto rate feature, we have simplified the job for you!
With this feature, you have the option to change auto-rate configuration at any state. Meaning, you can utilize it upon campaign creation, running and stopped status or for every new proof submissions. If you prefer using this feature upon campaign creation, you may visit this article on how to properly apply it.
In the campaign details page, you will be able to see the new Auto Rate feature as shown:
Ticking the checkbox for “Rate NOK if Less Than Minimum Success Score” would auto rate proofs as NOK (Not Satisfied) that did not match the enabled test case questions. On the other hand, if you tick the “Rate OK if Higher or Equal Minimum Success Score”, proofs will be auto rated as OK (Satisfied) only if all the enabled question’s test cases match with workers’ answers/proofs.
Once you tick any of the Auto Rate Tasks options, a confirmation pop-up box will appear acknowledging your choice.
In order for you to enable Test Case Questions, in the Campaign details page, you need to click the Manage Test Cases button. You will see this at the bottom of the page:
Then tick the checkboxes for Question Names of the correct answers. Please note, however, that when you choose “Rate OK if Higher or Equal Minimum Success Score”, you will have to manually rate the proofs if the answers can be one or two of your enabled test case questions.
Like the previous Auto Rate Task feature, Test Case file is also needed for this feature to function. Click the Download Test Case file button for you to be able to add the test case answers. Then upload.
Sample Downloaded test case file may look like this. Since our sample campaign requires checkboxes for workers to answer the campaign’s questions, correct possible answers can be True or False (case insensitive).
Once this test case file has been uploaded, auto rate feature will now take effect. If you have chosen to auto rate NOK proofs only, proofs that did not match your enabled test case questions will be rated as NOK or Not Satisfied and a new position will be open for another worker to complete and your campaign will resume until the new position is utilized. If you want to see the number of NOK ratings, choose “View 2” rating option and click the NOK tasks from the dropdown selection.
You will then be able to see the submission of workers by clicking the Template View button:
For manually rating tasks, you have 4 options to choose: Rate Tasks View 1, Rate Tasks View 2, Rate Tasks by CSV, Rate Tasks by ID:
For more information on how to manually rate tasks, you may visit these blogs:
Checking and Rating Submitted Tasks on TTV
Employer’s Guide to Task Rating and Revising through Submitted Tasks