AI literacy is essential for students to cultivate, and there are activities and assessments you can use to help them do so. Critical AI literacy—that is, the practice of assessing and evaluating AI and its outputs--is especially important for preparing students to cultivate the expertise necessary to be effective within a specific discipline.
To cultivate critical AI literacy, you can follow the specific activities in the frameworks listed in the AI literacy page. You can also focus on helping your students develop a more critical stance in general. This more general stance begins with the practice of resistant reading, or the practice of always trying to decode ulterior motives and never trusting what the text says on the surface. In the case of AI, this means never trusting what an AI says or does until the information has been validated.
Resistant reading of AI begins with assessing the LLM being used. LLMs have varying ideological biases, datasets, priorities, and capabilities. You can start by having your students determine their priorities. Are they concerned about the environment? Ethics? Ideological bias?
Critical Prompting
Once they've determined their priorities, have students interview a LLM (or two!) They should start with general questions and gradually move to more specific prompts. The following suggestions can help them achieve specificity:
Clarity and Specificity: Provide as much details as possible. Instead of asking, “tell me about the history of cutlery,” ask, “tell me about the history of the fork and the cultivation of table manners in the 17th century.”
Role-Playing and Personas: Assume a persona to help the AI better understand how to answer your queries. Example: “Act as a senior chemical engineer and tell me how to…”
Context and Constraints: Provide all relevant background information. Also, give the LLM examples of the desired output ( a tactic known as"few-shot prompting"), details about the target audience, or specific constraints to follow, such as "Do not use jargon" or "Keep the response under 100 words." Including constraints helps the model avoid irrelevant or overly verbose answers.
Actionable Verbs: Start the prompt with strong, action-oriented verbs like “summarize,” “create,” “compare,” “explain,” or “generate.”
Iterative Refinement: Start with a simple, clear prompt and refine it based on the initial output.
Activities and Assessments
Critical AI literacy can be cultivated through a number of activities. Your discipline will determine which activity is right for your students, but the following suggestions can give you some ideas:
Develop a prompt that asks students to evaluate AI output, either comparing it to their own work or models provided by you. Supplying a rubric they can follow will make this activity more effective. Include a series of follow up questions that ask students to reflect on what the AI did or did not do well.
If the AI did everything well, then ask students to reflect on what purpose the knowledge required to answer the question correctly serves. Why should one have certain skills or knowledge if AI can do it? You can sharpen this point by proposing specific situations: Would you want to take a flight piloted solely by AI? Or would you prefer an expert, a human pilot, working together with AI? Would you undergo surgery guided by AI alone? Or should a physician be present?
Have students compare the outputs of two LLMs. Ask them why it might be problematic to have answers that, while all correct, vary in their content.
Have students reflect on the importance of accuracy. Is it enough if something sounds or seems correct? What’s “good enough”?
Give students a prompt and have them compare the answers the LLM gives them vs what their peers receive. Is it problematic that they may receive different answers depending on their data trail?
Develop a rubric that provides basic guidelines for evaluating AI output in your discipline and establishing a foundation for cultivating expertise. Here is a sample rubric for a chemical engineering course.