Final project requirements

Submission links¶

• For the final report: click here

• For the presentation slides: click here

• To sign up for a technical report time slot use this Github Issue.

Introduction¶

Overview¶

In the final project, you will apply some of the statistical methods learned throughout the course to a real-world problem in Earth and Atmospheric Sciences of your choosing.

First, you will select a topic and corresponding dataset and get those approved by the professor.

Then, you will conduct the required data analysis tasks listed further below.

You will be responsible for three things:

1. A final report, in the form of a self-contained, fully executable Jupyter Notebook.

2. A lightning (5-minute) presentation to the class

3. A 15-minute technical interview with the professor about the project

Grading¶

Consult the syllabus for details.

Topic and dataset¶

You are welcome to use the Central Park dataset we’ve been using in class.

You’re also welcome to use something else. It’s totally fine if this overlaps with work for another course or your own research (for me at least; consult your other professors and/or advisors if you’re unsure).

If you have a topic in mind but not a dataset, try explaining your idea to an LLM such as ChatGPT and ask it for recommendations of datasets---and insist that it provides you links and verifies that they are real! If that doesn’t help, email me or come see me right after class to discuss.

Your dataset needs to be complex/big enough for the results to be non-trivial. So, for example, a dataset with only 100 points would not be enough.

But I recommend keeping the total size below ~100 MB so that your laptops/Colab can handle the calculations without them being really slow.

The Central Park weather station dataset is a good example: long timeseries (>56,000 points) of multiple variables (so $\geq$500,000 points total), but the file is still only 3.9 MB.

It doesn’t have to be timeseries data, but there does need to be at least two different variables or locations or etc. Otherwise, you won’t be able to analyze relationships (correlations etc.)

Some online data portals you could check out:

• NASA: https://data.nasa.gov/

• NOAA climate data: https://www.ncdc.noaa.gov/cdo-web/datasets

• National Centers for Environmental Information: https://www.ncei.noaa.gov/

• CMIP6 (climate models): https://esgf-node.llnl.gov/projects/cmip6/

• EPA: https://www.epa.gov/data/environmental-dataset-gateway

• data.gov: https://data.gov/

Final report¶

Due date: Friday, December 12th, by 11:59pm ET¶

How to submit: Google form here¶

Overall scope¶

There are two goals of the final report:

1. Demonstrate your mastery of select concepts we have learned in class, by applying them to your dataset in a way that is in service of the scientific question you seek to answer.

2. Use those calculations to actually make some progress toward answering your chosen scientific question.

Note that you will not be directly graded on the scientific outcomes. It could be that your analyses generate statistically unclear signals, even though you did them correctly. That still is progress: it’s often just as important to know what things are not related as what things are related.

Report format: a Jupyter Notebook¶

You will write the final report as a Jupyter notebook file, with your text as Markdown cells and your analysis code all included as code cells. See sections further below for style and formatting guidelines.

What you will submit: the notebook itself plus a freshly executed version of your notebook exported to HTML¶

The steps for submission are:

1. Create your report as a Jupyter Notebook.

2. Once your report is 100% ready, reset your Jupyter kernel and re-run the whole notebook from start to finish. (There is an option under the “Kernel” drop-down that does this for you: “Restart Kernel and run all cells”)

3. At that point, export it to an HTML file using Jupyter’s built-in exporting features.

4. Upload both the original notebook file (.ipynb) and the HTML export (.html) following the link provided.

Notebook contents¶

Like the homework assignments, the notebook will be a mixture of Markdown cells containing mostly text and code cells that perform your analyses and generate your plots.

Notebook structure: same as a normal written report would be¶

Though it is written as an interactive Jupyter Notebook, your final report should be organized and read as if it were a standard scientific report written in say Microsoft Word or LaTeX. This means it should:

• be organized into labeled and numbered sections and subsections (Use hash-signs in markdown cells for this: starting a line with # makes it a top-level heading; ## 2nd level heading, ### 3rd level, etc.)

• be written in complete English sentences, organized into paragraphs and sections.

• be free of grammatical, spelling, or other similar errors.

Perhaps a useful way to think about it is: imagine that the code blocks were stripped out, and only their outputs kept. The resulting document should look and read basically like an old-school printed final report would look and read.

Code blocks: excess code, style, commenting, etc.¶

Your code should be pruned down to only the lines required to generate the values you report and plots that you generate. All other blocks or lines of code should be removed! Otherwise it makes understanding and ultimately grading your code much more difficult.

Adding explanatory comments or adhering to style conventions is less important. Of course, you are encouraged to include helpful comments as appropriate and to use style-enforcement tools such as black (which you can configure to automatically run your whole notebook via the jupyterlab_code_formatter tool.

Required calculations to incorporate¶

The following calculations must be executed, presented, and described:

From Intro and numeracy:

• thorough explanation of the dataset: source, time span, location, physical quantities being observed, instruments used to measure them

• any other salient metadata: e.g. changes in instrumentation, change in location, calibration issues, spatial coverage

From Descriptive statistics:

• mean and median of at least one key quantity

• range, IQR, sample variance, sample standard deviation of at least one key quantity

• skewness and kurtosis of at least one key quantity

From Data visualization:

• at least one each of: histogram, boxplot, scatterplot, and timeseries (unless the data are not defined in time)

From Probability theory:

• at least two empirical unconditional probabilities

• at least two empirical conditional probabilities

From Probability distributions:

• empirical PDFs and CDFs

• a fitted parametric distribution

• a block maxima/minima analysis and corresponding GEV fit

From Hypothesis testing:

• at least one $t$ test of differences in means between two samples

From Linear regression:

• at least two correlation coefficients

• at least one variable modeled by linear regression on another variable

From Time series:

• discussion of how the decomposition into deterministic and random components of a time series would be applied to at least one key variable, even though you don’t have to actually perform that decomposition

• calculation and discussion of the autocorrelation function of at least one key variable

From Spectral analysis:

• a periodogram computed, plotted, and discussed for at least one key variable

• a running average applied to your periodogram

OPTIONAL EXTRA CREDIT, for up to 20% of the total report grade: From neural networks and machine learning

• A functional feedforward neural network with one or more hidden layers trained on your data. Must include a 2-3 paragraph discussion of the problem it’s solving, how you trained it, choices of hyperparameter values, etc.

Grading¶

(credit: copied nearly verbatim from Teaching Statistics: A Bag of Tricks by Andrew Gelman and Deborah Nolan)

Rubric¶

The table below is a competency matrix for this report. The first column describes each critical task for the assignment, and the 2nd, 3rd, and 4th columns respectively describe what work in that task would constitute Needing Improvement, Basic Competency, and Surpassed Expectations.

Assigning points¶

Basic competency in all five categories results in 75 points.

Five points are added for each task in the Surpassed category.

Similarly, five points are deducted for each competency in the Needs Improvement category.

As such, the maximum possible score is 100, and the minimum possible score is 50.

In-class presentation¶

Each student will present a “lightning” oral presentation to the class explaining one interesting result from their final project.

Logistics¶

SUBMISSION LINK for slides here.¶

Presentation date: Monday, December 15th, 2025 during normal class time.¶

The order of presentations will be assigned randomly.

Submission deadline for slides: 11:59pm Sunday, December 14th, 2025¶

The professor will download the submitted slides to his computer the morning before class, and everyone will use the same computer to present (rather than each person trying to connect their own computer to the A/V system one after the other).

Format¶

Lightning style means the following:

• Total duration: 4 minutes

• Presentation: 3 minutes

• One question from the professor: 1 minute

Presentation requirements¶

Scope: tell us one interesting thing you found.¶

Whereas the written report for this project is meant to be fairly exhaustive, where you document all the important analyses that you performed, an oral presentation has to be more targeted. Four minutes will fly by! DO NOT attempt to summarize your whole project.

Instead ask yourself: if you had to pick just one thing that you want to convey about your project, what would it be? Then build your talk around that.

Slides: 3 or 4 slides, no more or less.¶

These can be PDFs, Powerpoint, Keynote, or Google Slides.

1. Title slide

2. Project background: what’s the specific discipline within Earth science, what’s the specific dataset and variable(s) you’re analyzing

3. The one result you’re sharing

4. (optional) Additional material about that one single result

Recommendations¶

Slides¶

Some guidelines:

• Less is more.

• Make each slide do one thing, not multiple things.

• Make each slide’s title a complete sentence that summarizes the main point you want the slide to convey.

• Some text is helpful, but usually people include too much. Boil it down to the essentials.

• Plots: describe in words every single image and table you include. This is for accessibility, but also because almost always audience members can’t tell as fast as you think they can what it is you’re showing.

• Make all text, plot labels, plotted symbols, and images big enough that everyone in the room can read them.

• Give your slides some breathing room: a slide that’s totally full with text, multiple plots, etc. is overwhelming and results in less information being effectively conveyed than if you had less.

• Related, dispense with “slidejunk”: you don’t need slide numbers, logos, the date, etc. on every slide. (Except for a logo of your institution on the title slide, you don’t even need these anywhere!)

• If the professor’s own slides for this class fail to meet these recommendations sometimes, well, “Do as I say, not as I do” ;)

Delivering the presentation¶

• Try not to worry! Public speaking can be intimidating, but especially in this setting everyone, the professor and the other students, are there to support you and learn from you.

• Practice the talk at least once ahead of time with a timer. Make sure that you’re within the time limit. Nobody likes a talk that goes way beyond its allotted time; it’s rude to the audience and the other presenters.

• Don’t be afraid of silence. For the audience, it’s actually a huge relief when a speaker takes a few seconds between slides or to take a sip of water. It helps the audience take a second to gather their thoughts.

Answering questions¶

• It can be helpful for everybody, yourself included, to repeat the question back to the person in your own words for two reasons: (1) you make sure everyone in the audience heard it. (2) You make sure that you interpreted the question correctly.

• Once you’ve confirmed you understand what they’re asking, take a second (or a few)! There’s no need to answer as soon as the last word is out of their lips.

• If you don’t know the answer to a question, that’s OK! Take a few seconds to think hard about it, and then just give it your best shot. Or, if you’re truly unsure, it’s not a bad thing to answer “That’s a good question, and I’d have to think about it more.”

Grading¶

Your presentation will be graded based on the following:

1. Narrative quality: do you tell a coherent scientific story about one specific noteworthy result from your project? Or do you try to pack in too many different things?

2. Science quality: are the arguments, calculations, and plots presented valid? Or do they include errors or other problems?

3. Slide quality: does each slide convey a message in service of your story? Is there enough text, plots, etc. on each slide to convey that message? Is there too much on each slide for the audience to digest? Are the fonts big enough?

4. Length: did you complete your presentation within the time limit? Or did you go over, or alternatively way too fast?

5. Answering questions: do you make a good-faith attempt to understand and address each question? Do your answers cohere with what you presented?

Your grade will be based on a rubric nearly identical to that for the final report provided above, but using each of the five categories immediately above.

Technical interview¶

Overview¶

You will sit down with me for approximately 15 minutes in my office, with a live running Jupyter or Colab session with your final report, exactly as you submitted it.

I will ask you various questions about your report. Some of these will be conceptual, some more technical.

This will include requests to compute additional things on the fly.

Motivation¶

LLMs have gotten very good at analyzing data---at least if you pick the right model and prompt it well. As such, it’s possible to have ChatGPT etc. generate much or perhaps all of your final project report with little effort or understanding on your part.

This technical interview will enable me to evaluate how well you yourself understand the dataset, the analyses you performed, their implications, etc.

As a secondary motivation, technical interviews are commonplace in scientific and computing jobs. (And they usually are more like an hour rather than 10 minutes.) So this will give you some experience with that format; hopefully that will help you prepare for future technical interviews.

Time slot sign ups¶

Sign up for a timeslot using this Github Issue.

What to do in advance¶

1. Most importantly, make sure your final report notebook is working, i.e. that when you restart your kernel and run all cells, it runs start to finish without crashing.

2. If you have a large dataset and/or lots of calculations, it could take several minutes for the whole notebook to run. If it takes more than five minutes, then please come with the notebook session already live and with all cells successfully run.

3. The notebook should not be substantially altered from the one you submitted for your final report, unless after submission you discoverd an important error or problem that you then have corrected.

What you’ll be asked¶

You will be asked five distinct questions. The first one will be fairly simple. For example, “Show me the line(s) of code that compute and display the linear regression slope.”

Subsequent questions will vary in difficulty. Some will be easier and others more difficult than the first one.

An example of a more difficult question might be “How sensitive is the result of your t test to the specific two samples you’ve chosen?”

• A good response would be to find your code that computes the t test, copy it into a new cell, and modify that new cell to compute the t test on two samples that are somewhat different from your original ones.

• An OK response would be not being able to get the new code working correctly, but being able to describe out loud in general how you’d go about it.

• A bad response would be not knowing which of the code you submitted computes the t test and/or not understanding t tests well enough to be able to answer the question coherently.

Some questions, like the example just above, will entail you running some new code, mostly by modestly adapting some part of your existing code. Other, conceptual questions will not require running any code.

I will tailor the questions for each student to the final report notebook that you submitted. That’s why above I ask that you leave your notebook mostly unchanged from the submitted version.

Grading rubric¶

This is similar to the rubrics for the report and the presentation. You start with 75 points. Each correct answer gains you 5 points, a partialy correct answer gains you 0 points, and an incorrect answer loses 75 points. So the minimum score is 50 and the maximum is 100.