weeknotes: new thesis direction, creative optimization

Back to school!

thesis

A big thing happening this week is pivoting my thesis. For the last few months, I’ve been warily explaining my thesis idea around “parameter space exploration”. I’m a lot more confident about my (very rough) working title of “birdwatchers’/naturalists’ guide/field notes to (computational art/parameteric systems/etc)”.

In birding, you can only really move yourself around, be quiet, watch, and wait (and, in the longer term, encourage a healthy environment for birds). It makes for a counterweight to words like “control” or “optimization” that show up in my work.

I’m not entirely sure what the thesis would look like yet, but I could reflect on attentive observation, the senses, field notes, dynamics, and which abstractions to use in the context of creative coding. Maybe I could spend some more time on generative systems around feathers, rocks, and leaves. To me, the ideal tools for birding are binoculars and Merlin, which just give you more abilities to see and understand things.

So I’m sorting out how exactly this thesis will look and trying to find the right resources to help me crystallize it.

creative optimization

Ah, another week, another rabbit hole. I spent a lot of time this week trying to get a differentiable system to replicate a physical system, but even when I took care of a lot of bugs, it wasn’t producing very good results compared to the existing greedy approach.

So this approach is tabled for solving the main project. That said, I’m trying to make some viz using the system. It’s pretty, and I think there’s something there: like… I’m rendering SDFs on my GPU… but via JAX instead of shaders??

I’d already pulled out a bunch of hyperparameters into namedtuples for some tuning (which continues to be a really useful way to learn what different parameters do!) When I switched to more creative optimization, I would swap out the target image over time, slide different hyperparameters, and scale different parameters’ gradients by different amounts.

However, it is agonizingly slow, running at like 0.1fps. Having it be real-time could be really cool, like live coding targets and adjusting the parameters on the fly! I’d need to do some performance profiling, but a wild idea is to skip JAX and build it directly in webgpu and hardcoding the derivative..

I also haven’t looked at how stream diffusion or other real-time optimization things look. I don’t really want to rebuild something that looks the same as what already exists!

But anyway, I wanted to get one proof of concept out, and then take a step back and see what I want to do with it.

birds

I finished the goldfinch and started on a chimney swift!

I used to have different debug modes I could manually switch into, but I thought it would look interesting to show them all at once. This view turns out to be quite useful for building a new bird!

Something that came up was whether to lean into repetition or randomness in the little tuft of black on the goldfinch’s head. Repetition is an interesting stylization that I might return to, but for now, I went with randomness to match the robin.

I started on the next bird, a chimney swift. I thought it would be a lot about a completely new kind of wing, and now that I’m getting to the coloring, I realize it’s got a little white chin that blends into its body, these hawk-like eyebrows, and pointed tail feathers. Close-up photography reveals beautiful details on the feathers, too.

Chimney swifts are fascinating! Through the summer, you hear them screaming overhead. They seem like they’d be similar to tree swallows, but they’re actually more closely related to hummingbirds. Swallows often perch, but swifts barely have legs and have to cling to vertical structures (like chimneys) to take off.

Their numbers are also declining alarmingly, in part because of fewer open chimneys (or originally their hollow tree trunks) as well as declining insect populations.

etc

Ended the week with a bike ride to Watertown and Mt Auburn (it was too warm for a longer ride, but still got to go somewhere new), and a few hours of monotype printmaking with Coco! So far, for classes, I’m planning to take Vision in Art and Neuroscience, because it seems useful to learn what we know and don’t know about vision and perception, and the class has a cool lineage from Bauhaus to the Center for Advanced Visual Studies.