“Can I benefit from using AI?” Assessment Questionnaire
The following are key assessment questions to consider that will help you determine if you would benefit from an AI solution for your agency use case. After completing the assessment, determine your total score to gain insight into the possibility of a substantial return on investment (ROI) from the integration and implementation of the proposed AI solution. Increased ROI can also mean increased effectiveness and/or efficiency. Also, be sure to consider the savings gained based on the time and level of effort needed to achieve the end state goal.
Please NOTE: This is a notional assessment, and the level of importance associated with each question may differ based on the agency and the specific use case. Points are based on the Attribute Importance Rank (with suggested weights).
1. Does the use case clearly, and accurately describe the problem to be solved?
2. Does the use case accurately outline current processes and current state in place?
3. Does the use case align the goals and objectives with desired outcomes?
4. Can you use an LLM to ask questions about areas that you are not considering? And LLM-enabled web search? Using web search, is there an open-source solution/model that would partially solve the problem at this time?
5. Have other technologies successfully been applied to address elements of the use case? (Could you somewhat solve your use case with an existing solution?)
6. Does the use case contain sufficient information to determine the extent to which AI is required?
7. Has sufficient representative data been identified for the use case?
8. Are the data required and available, accessible, and accurate?
9. Is the data from the use case annotated, curated, and tailored to the use case? (Does the data contain metainformation?)
10. Would intelligent automation reduce operational burden? (Note: This is to determine if only RPA is needed.)
11. Are there repetitive manual tasks that can be automated beyond traditional scripting?
12. Is there a predictive need for the use case? (Assumptions and testing made based on prior data)
13. Are the data fit for purpose (descriptive modeling) and are they operationally relevant (predictive modeling)?
14. Is there representative enough information in the data to support training an AI model which is aligned with the use case?
15. Is the use case high risk based on regulatory concerns?
16. Does the use case contain other risks –such as a potential for data bias? Are the data used for the intended purpose? (In the data, algorithms, or aggregation process)
