+ What is the AIQ? And how will it promote AI’s core mission?

The AIQ or “Arbitrator Intelligence Questionnaire” is a feedback questionnaire designed by AI to facilitate systematic collection of information about arbitrator case management and decisionmaking.

AI’s core mission is to promote fairness, transparency, accountability and diversity in arbitrator appointments. By making systematically gathered data about arbitrator decisionmaking more equally available, the AIQ will help level the playing field in arbitrator selection and make the entire process more transparent. Meanwhile, more information about arbitrators will provide market-based accountability for arbitrators and allow newer and more diverse arbitrators a meaningful opportunity to build their reputations based on actual performance.

+ How was the AIQ developed and what is its current status?

AI developed the AIQ in coordination with the Penn State Survey Research Center. Earlier versions of the AIQ were pilot tested and commented on by several external advisors and commentators, who represented a range of stakeholders and areas of expertise.

The AIQ was launched in Singapore in cooperation with SIAC on 1 June 2017. The AIQ is currently available on AI’s website.

+ How will the AIQ be implemented?

Since its formal launch on 1 June, the AIQ is open for all parties, lawyers, and third-party funders to fill out after the conclusion of each arbitration in which they are involved. To promote more systematic completion of AIQs, AI is entering into Collaboration Agreements with various arbitral institutions. Under these agreements, institutions will forward the AIQ to parties and lawyers at the end of each arbitration in exchange for access by institutions to AI data about arbitrators. AI is also inviting parties and law firms to participate in The Arbitrator Intelligence Pact, under which they commit to supporting AI’s goals of fairness, transparency, accountability, and diversity by submitting AIQ responses regularly at the conclusion of arbitrations.

+ Who can complete the AIQ?

AI aims to have a range of participants—including parties, their in-house counsel, and their outside or external counsel, and third-party funders—routinely fill out AIQs. Multiple responses from different participants in a single case are welcome as they will facilitate increased quality control and more nuanced data assessment. Arbitrators, arbitral institutions, and arbitral secretaries, however, are not invited to participate to avoid disclosure of any confidential information.

+ Why are there 2 Phases to the AIQ?

The AIQ is divided into two phases to ensure that responders can provide feedback in 15 minutes or less per phase. Phase I concentrates on objective background information about the case, and can be completed by anyone who has access to the award or case file. Phase II contains questions that are evaluative and, in some questions, request subjective assessments. Phase II should be completed by an attorney or party who actively participated in the case. Certain background information from Phase I questions is used to prefill the questions in Phase II to make it even easier to complete Phase II.

+ How does the AIQ protect the parties’ confidentiality?

To protect parties’ confidentiality, the AIQ does not seek information that would readily identify the case or the parties, but only key data that will facilitate assessment and analysis of arbitrator decisionmaking and case outcomes. For example, the AIQ asks for the date of filing, the industry in which the dispute arose, the date of the award, and arbitrator names. The AIQ does not ask for parties’ names or the names of law firms or lawyers. For quality assurance purposes, responders will be required to register in order to complete an AIQ, but their identities will be kept confidential and will not be published or otherwise connected with any AIQ responses.

+ How does the AIQ ensure that information provided will be relevant, accurate, and reliable?

The quality of information generated by the AIQ will be key to its success. For this reason, the AIQ was specially designed in consultation with experts in survey design and after extensive public consultation.

At a more substantive level, the AIQ includes questions that will facilitate qualitative assessment of responses and filtering of data from AIQ responses. For example, responders must identify their perspective (in-house counsel, outside counsel, etc.), their level of experience, and whether the award was more or less favorable than expected, etc.

Responses can be assessed against the background information about the case, such as the amount in dispute, the governing law, the seat, or the industry in which the dispute arose. Responses can also be assessed for their consistency with the objective information derived from related responses on a particular topic.

+ How does the AIQ ensure that feedback is fair to arbitrators?

The AIQ has a number of features designed to ensure not only the accuracy, but also the fairness of the information it provides. While the identity of responders is confidential, responders are required to register and verify their identities before completing the AIQ. Before any AI Report about an arbitrator is published, arbitrators must give express consent after being provided with access to the data collected regarding them. Arbitrators will also have an option of withdrawing consent, in which case AI will no longer offer Reports on that arbitrator.

Meanwhile, AI is working to establish editorial policies to ensure that only AIQ responses that meet minimum indicia of reliability will be incorporated into AI Reports. Open-text feedback provided through AIQs will similarly be vetted through established editorial policies and generally published in summary form, similar to existing practices used by the International Mediation Institute and the Almanac of the Federal Judiciary for mediators and U.S. judges. AI is also exploring other means of ensuring that feedback is fair to arbitrators.

+ How will information from the AIQs be made available?

Once sufficient data are collected from responses to the AIQ, those data will be made available (usually for a fee) to parties, counsel, institutions, and arbitrators through “AI Reports.” AI and Wolters Kluwer have entered into a collaboration to make AI Reports available. Although the specific features and content of AI Reports are still being developed, they will provide data on how prospective arbitrators compare with other arbitrators on a range of data collected through the AIQ, including historical ordering of interim measures and document production, interpretative methodologies in construing contracts and treaties, and timeliness in issuance of awards.

+ Why do I have to pay to get information back if I am contributing AIQs?

Information collected through AIQs cannot be made directly available in an open access database without serious risks to data security and confidentiality. For these reasons, and to optimize the utility of information collected through the AIQs, it will be analyzed by Arbitrator Intelligence and published as Arbitrator Intelligence Reports (AI Reports). While the AI Reports will be available for a fee, Arbitrator Intelligence is a non-profit organization. Revenues it earns in fees from AI Reports will be used to cover the costs of generating the AI Reports, including the cost of specialized software to generate the reports, or directed toward related projects to promote transparency, accountability, and diversity. Pricing for reports has not yet been determined, but will include significant discounts for those who contribute AIQs. More information about AI Reports will be available soon.