Persistent Link:
http://hdl.handle.net/10150/105417
Title:
Evaluation of Algorithm Performance on Identifying OA
Author:
Antelman, Kristin; Bakkalbasi, Nisa; Goodman, David; Hajjem, Chawki; Harnad, Stevan
Citation:
Evaluation of Algorithm Performance on Identifying OA 2005-12,
Issue Date:
Dec-2005
URI:
http://hdl.handle.net/10150/105417
Submitted date:
2005-12-19
Abstract:
This is a second signal-detection analysis of the accuracy of a robot in detecting open access (OA) articles (by checking by hand how many of the articles the robot tagged OA were really OA, and vice versa). We found that the robot significantly overcodes for OA. In our Biology sample, 40% of identified OA was in fact OA. In our Sociology sample, only 18% of identified OA was in fact OA. Missed OA was lower: 12% in Biology and 14% in Sociology. The sources of the error are impossible to determine from the present data, since the algorithm did not capture URL's for documents identified as OA. In conclusion, the robot is not yet performing at a desirable level, and future work may be needed to determine the causes, and improve the algorithm.
Type:
Technical Report
Language:
en
Keywords:
Web Metrics; Scholarly Communication; Economics of Information
Local subject classification:
Open Access; Open Access Advantage

Full metadata record

DC FieldValue Language
dc.contributor.authorAntelman, Kristinen_US
dc.contributor.authorBakkalbasi, Nisaen_US
dc.contributor.authorGoodman, Daviden_US
dc.contributor.authorHajjem, Chawkien_US
dc.contributor.authorHarnad, Stevanen_US
dc.date.accessioned2005-12-19T00:00:01Z-
dc.date.available2010-06-18T23:25:03Z-
dc.date.issued2005-12en_US
dc.date.submitted2005-12-19en_US
dc.identifier.citationEvaluation of Algorithm Performance on Identifying OA 2005-12,en_US
dc.identifier.urihttp://hdl.handle.net/10150/105417-
dc.description.abstractThis is a second signal-detection analysis of the accuracy of a robot in detecting open access (OA) articles (by checking by hand how many of the articles the robot tagged OA were really OA, and vice versa). We found that the robot significantly overcodes for OA. In our Biology sample, 40% of identified OA was in fact OA. In our Sociology sample, only 18% of identified OA was in fact OA. Missed OA was lower: 12% in Biology and 14% in Sociology. The sources of the error are impossible to determine from the present data, since the algorithm did not capture URL's for documents identified as OA. In conclusion, the robot is not yet performing at a desirable level, and future work may be needed to determine the causes, and improve the algorithm.en_US
dc.format.mimetypeapplication/pdfen_US
dc.language.isoenen_US
dc.subjectWeb Metricsen_US
dc.subjectScholarly Communicationen_US
dc.subjectEconomics of Informationen_US
dc.subject.otherOpen Accessen_US
dc.subject.otherOpen Access Advantageen_US
dc.titleEvaluation of Algorithm Performance on Identifying OAen_US
dc.typeTechnical Reporten_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.