Caveon Test Detective
1.
General concept - We are ready to provide self-service web-based statistical analyses for answer-copying or collusion on multiple-choice tests to the general public. Please indicate how you might use such a service (please check all that apply):
Verify known and observed testing irregularities
Create a report of the degree of potential cheating, but take no direct action
Monitor overall level of cheating, and revise tests policies as a result
Take specific actions for instances of extremely similar test results
Other  
2.
Frequency of use - How often would you use this service to support the administration of your multiple-choice tests?
Weekly
Monthly
Quarterly
Every six months
Once a year
Never
Other  
3.
Value of service - How would this service increase your confidence in the results of tests that you give?
I would increase reliance upon the test scores
I would increase the frequency of tests
I would use results of potential anomalous data in making pass/fail decisions
I would require retesting when the tests were too extreme to be due to chance
Other  
4.
Number of tests - How many tests do you typically administer that you would analyze with this service or product?
Less than 20
Between 20 and 50
Between 50 and 200
More than 200
Other  
5.
Ease of data submission - Several data formats could be supported. We expect that the responses for each test will be provided on a single line in the file, with an additional line that provides the answer key. Which data formats would be most convenient for you?
Excel spreadsheet
Comma-separated values file (CSV)
File from a statistical package (such as SPSS or SAS)
Database (such as MS Access)
Data upload from a grade reporting system (such as Blackboard)
Other  
6.
Item types - We have envisioned that the service is most readily suited or 4- and 5-choice multiple-choice items. Which other item types and formats do you typically use?
True/False
Matching
Grid-in items
Free response
Multiple select
Essay
Other  
7.
Amount of cheating - How often do you typically deal with suspicions or allegations of answer-copying or collusion on your tests?
Weekly
Monthly
Quarterly
Every six months
Once a year
Never
Other  
8.
Responsiveness - How quickly would you expect or need to receive the analysis after submitting the data?
Almost immediately
I can wait 30 minutes
I can wait 2 hours
I can wait overnight
I can wait up to 2 days
Other  
9.
Report features - Which types of information would be most useful to you in the statistical report?
Side-by-side illustrations of similar answers
Pie charts of observed amount of similarity and expected amount of similarity
Bar charts with indications of extremeness of the similarity
Descriptive text with interpretations
Qualitative ranking of similarity (e.g., low, medium, high)
Probability statistics of the similarity
Other  
10.
Participation in Beta - Under what conditions would you be most willing to participate in evaluating a free, introductory beta version of the service or product?
Not interested in participating in a beta test evaluation
I want to participate, and provide immediate feedback via e-mail
I want to participate, and provide feedback only with a follow-up survey
I want to participate, and am willing to discuss my experience on the phone
Other