Special lab: Generative AI tools: using—and not using—them to optimize your learning#
CCNY EAS 42000/A42000, Fall 2025, 2025/10/22, Prof. Spencer Hill
SUMMARY: Generative AI (genAI) tools, most of all Large Language Models (LLMs) such as ChatGPT, are incredibly powerful and rapidly growing even more powerful. Here we’ll reflect on how to use them as a tool to accelerate your learning rather than as a crutch that degrades your learning.
Terminology#
Generative AI (genAI) refers to computer programs that have been trained using machine learning techniques to generate outputs in response to user requests.
Probably the most familiar type of genAI tool is the Large Language Model (LLM): things like ChatGPT, Claude, and Gemini which accept your text and respond to it (usually/mostly) with text. These all started out purely text-based, but now they can also
GenAI is a more general term than LLM, because genAI includes things like Midjourney that only output images rather than text, or that output video, or audio. But for our purposes, that distinction doesn’t really matter, and we’re mostly concerned with LLMs, so I’ll be pretty lax about using “genAI” and “LLM” interchangeably. In fact, I’ll even sometimes just use “ChatGPT” as a shorthand for all LLMs.
Big-picture concerns with generative AI#
There are big-picture concerns one might reasonably have about generative AI tools in general:
AI alignment: Will we all be turned into paperclips? Jokes aside, how do we ensure that AI remains aligned with our values and does what we want it to? It’s not impossible after all that the models end up deciding to do what they want and/or agree to do very bad things.
Ethics (training, job displacement): Should the creators of the content the models are trained on be compensated? If so, how?
Environmental impact: energy required to power all the data centers, among other concerns
There are some people that, based on one or more of these factors and/or other factors, decide not to use genAI tools whatsoever. Having said that, we’ll proceed to considering how you, as a college student, should approach their use.
A conceptual framework: “What are all the ways that AI influences my learning?”#
First, let’s get past the “LLMs are good/bad” false dichotomy.
They are both, neither, and in between, and a million other things. It all depends on how you use them!
Second, let’s also get past the adversarial approach, teacher as cop and student as naughty boy that can’t resist the “seductress” that is ChatGPT; in other words, the “cheating/preventing-cheating” dichotomy.
Even if you’re not breaking any rules by using it, you may well be cheating your future self by harming your own learning outcomes.
That said, prior experience convinces me of two things.
A decent fraction of students will use it in any context it is available, even if it is nominally forbidden. (I.e. to cheat.) This is why 40% of your semester grade in a methods class will come from in-class, closed note exams, and a large portion of your final project grade will be from the 1-on-1 “technical interview.”
The default mode of using AI—whether or not it would count officially as cheating—is mindless, uncritical: genAI as a crutch; copy-paste HW question, copy-paste LLM answer, etc. Whether or not this is “official” cheating, it is cheating yourself.
This is especially a bummer because, arguably, domain expertise will become more important, not less, as AI continues to advance. Why? Because it is those people who understand something most deeply that can (a) verify the accuracy of the AI outputs, and (b) that can use that expertise to ask the most interesting and important questions of the AI.
(For example, I use ChatGPT all the time now to help me construct simple analytical “toy” models, meaning one or a few simple equations, as a first step in developing new research ideas. For a particular example, I will be doing that again soon regarding temperature trends in the Central Park dataset we’ve been using, how those compare to those from nearby stations at the airports, and what role the vegetation in Central Park plays in those differences.)
So the task before us is: how to break out of those patterns? How to harness the incredible power of these tools and still develop the knowledge and skills you need for the future?
Types of risk associated with LLM outputs#
To start, it helps to be explicit about the different ways that LLMs can lead you astray (c.f. Mollick and Mollick 2024):
Confabulation risks: a.k.a. “hallucination” a.k.a. “making stuff up.” Depending on the context, these can be very easy or very difficult for you to spot. You are responsible for verifying all LLM outputs.
Bias risks: Based on their training data and their fine-tuning, LLMs can be biased in various ways: gender bias, other identity-based bias, political and viewpoint bias. This can be quite difficult to spot. For this course on statistical methods, such issues likely won’t be hugely important. Regardless, it’s important to be aware of this more generally.
Privacy risks: Assume that anything you feed into an LLM will be later used to train future generations of the same LLM or other LLMs. The legal status here is all shaky. Note that doesn’t necessarily mean that any humans will be reading your individual ChatGPT sessions.
Instructional risks: an LLM might teach you material in ways that differs from how it’s being taught in this class. E.g. using different notation or slightly different definitions. Separately, there’s the single biggest danger in my mind, of you using it as a crutch that replaces your learning rather than a booster that acclerates it.
Goals for this class relating to genAI use#
Goal 1: train yourself to always identify the point at which you lose the ability to confidently evaluate the LLM output’s accuracy#
It can be useful to consider two limiting cases here: extremely easy and extremely difficult questions.
If you ask an LLM (or any person for that matter) a simple arithmetic problem, of course you’ll be able to tell right away if they got it wrong.
Conversely, suppose you ask an LLM about a complicated topic you know nothing about. For example, some things I don’t have much expertise in are Japanese anime and quantum mechanics. So if I ask an LLM about either of those things, I will have to take its answers at face value.
These two limiting cases are easy to identify. But, as often, the challenge is that most of the time falls into a gray area between the two limits. That’s especially for you, a student, trying to learn new material: you’ve got some background knowledge on the topic, and maybe even think you understand it pretty well
Goal 2: identify cases where LLMs helped vs. hurt. vs. was a mixed bag regarding your learning#
Every student experience and be consciously aware of each of the following happening at least once for this class over the remainder of the semester:
An instance where using generative AI meaningfully advanced/improved your learning
An instance where using generative AI product meaningfully harmed your learning
In between: an instance where using generative AI was a mixed bag and/or it was hard to say how much it helped vs. hindered your learning.
These determinations will likely be hard to make in the moment! They will require some reflection and attempts to recall the knowledge you were trying to gain in the moment.