On nearly every indexing-related article on this blog, regardless of the subject, one or more people make an “arbitrator done-me-wrong” sort of comment. It is the most commonly expressed concern of indexers. But if you judge the overall state of indexing and arbitration by these comments or even by your own negative experiences, you may be in need of a little more straight talk.
Here’s What’s Really Going On
For starters, arbitrators are to be commended for working tirelessly on the hard and sometimes undesirable work that has resulted in the consistently high-quality indexes now on FamilySearch.org. The work of these volunteers is highly valued, and they can’t be recognized enough for their contribution. The fact that they occasionally make mistakes is to be expected. And it’s OK.
Remember, indexing and arbitration are volunteer activities. No one who does this work went to school to learn how to do it, and no one has a college degree or special certificate indicating that he or she is particularly qualified to do the work. If FamilySearch were to require that sort of certification of volunteers, we would all still be using microfilm to find our ancestors.
Indexing is a cause that people care about, and most people take it very seriously and do the very best work they know how. So let’s not assume that the isolated negative experiences we all have had with arbitration are somehow reflective of the whole effort. They clearly are not. That said, there are things we all can and should do to improve the quality of published indexes.
Indexers can read the instructions more carefully and take the time to make sure they are being methodical and precise. If they can’t read a name or find the instruction they need, they can share their batch with a more experienced indexer, or they can share it online in one of the indexing communities where lots of dedicated experts hang out. Indexers should take responsibility for doing their best work and not rely on arbitrators to correct their mistakes.
Arbitrators can improve by taking the arbitration self-assessment, studying and following the recommendations that have been given, and taking the time to be deliberate and careful. Despite their experience, they should read and fully understand the instructions for every project they arbitrate. Arbitrators who don’t index are generally also the ones who receive the most feedback about mistakes. Before arbitrating batches in any project, an arbitrator should first index enough batches of the project to understand its unique quirks.
FamilySearch can improve by striving to write clearer, noncontradictory instructions; continuing to encourage stake indexing directors, group administrators, and arbitrators to focus on quality rather than quantity; and working on solutions that will allow indexers and arbitrators to be more collaborative.
A (Last) Word about the Arbitration Results Number
You may have recently heard that the arbitration results number has been discontinued. After surveying many indexers, we learned that although a number of indexers gain personal satisfaction from the number and use it for positive reinforcement, a much larger percentage of volunteers feel the number is more punishing than encouraging. Indexers seeking to improve will still be able to review the work of arbitrators and see the arbitration results of a specific batch, but those numbers will no longer be combined into an overall percentage.
When FamilySearch began publishing this number for the first time in 2011, it was to answer requests from indexers who wanted a way to know how well they were doing. While the number gives quick feedback, too many people let it determine how they feel about indexing and even about themselves. Far too many have let this number persuade them that indexing is either too hard and beyond their capabilities or too frustrating and not worth continuing.
This number led one very accomplished indexer to write a letter expressing her frustration and the reasons she was giving up indexing. She indicated that indexing was, for her, a form of relaxation, but the stress from her arbitration results number made it no longer enjoyable. This feeling is tragic. Here was a person who was finding fulfillment and who was making a substantial contribution, and she gave it all up because of the arbitration number.
Now, in fairness, some bad arbitration may have been done on some of her batches, but that would have happened with or without the number. And since there was nothing she could do about the bad arbitration, it probably would have been better if she had never known about it.
By removing the arbitration results number, we lose something of value, but in our view and in the view of most indexers who were surveyed, we gain something more important and substantial. This comment from one survey respondent seems to summarize things fairly well.
“It is good to know if mistakes were made, but grading is a discouragement to me. There are so many factors involved with each individual document, so if one person feels it is one way and another person sees it differently, it does not necessarily mean the indexer is doing poor work. You spend lots of time trying your best to decipher the writing, and I always felt I did my best and was sure of what I put down. It was a bit of a blow to see the arbitration results, knowing how much time and effort I put in.”
What Happens When Something Is Arbitrated Incorrectly?
Many have asked about the effectiveness of the feedback button. We track the amount of feedback individual arbitrators get. When there is excessive feedback, FamilySearch representatives reach out to those arbitrators individually to provide them with additional help and training.
Beyond this, the FamilySearch.org search engine compensates for many common transcription errors (including missing diacritics, although that is no excuse not to index them properly). Future improvements will include capabilities such as making both A and B indexed values searchable, enabling users to make corrections in published collections, and potentially even matching volunteers’ skill level and personal research interests to specific projects. We intend to do everything we can to continue to find ways to avoid indexing errors and to correct them when they happen.
As always, grateful researchers the world over thank you for your hard work and your concern with the quality of the work you do. As you continue in your remarkable service, please do your best to understand the projects you work on, and then be patient with yourself and others as we all try to accomplish this magnificent task together.
You may also like: