weeknotes: active learning, Bézier curves, rust

Active learning

After some reading, I’m getting back into my thesis work.

A lot of interactive art could be thought of as exploring a parameter space and finding interesting corners to explore. But in practice, many systems have a lot of buggy or boring areas, which can disrupt free exploration.

As an initial proof of concept, I have a simple problem: posing the 2d bird drawing I’ve been working on. That would let me come up with some generative bird poses based on their foot positions and create a big generative collection of birds in lively poses. And if I get this proof of concept working, I can generalize it to things beyond crude character posing (patterns! animations!)

So for the bird project, I split out the parameters related to the pose (foot position or angle of the body) from what the bird looked like (colors, feather shapes, borb-ness).

The challenge of a bit of realism strikes again: am I modeling the nearly weightless, happily upside-down warblers, or am I modeling robins who stand up straight? On the other hand, the roundness of a bird could also technically be part of their pose, since it depends on how much they are floofing up their feathers. Sigh. So I’m pretending it’s a hypothetical songbird that can do it all.

(This is an ongoing issue where I’m more interested in an expressive/impressionistic/abstraction of a bird, and I don’t want to perfectly replicate a bird. But once I model something pretty well, the incorrectness starts to jump out.)

I dropped it into the tool I’ve been building, silverspot, which gives an interface to explore random samples. I used some intentionally loose constraints and saw what came out:

Oh dear, it is now a frog? Well, this demonstrates the issue of buggy outputs!

Normally, I would go in and try to manually modify constraints so that the bird looked okay. But I would probably be too conservative, and some creative poses would be removed.

So I’m trying a new approach! I added some simple automatic checks, like whether the legs, wings, and eyes are attached to the body. I can run through tons of examples without even rendering the bird (which is good because rendering does take a bit of time). If the basic check passes, I render the bird and then run it through other checks that require pixels (starting with the YOLOv8 “bird” confidence score).

Now I can run through 1000s of examples (only somewhat slowly). The especially snazzy active learning part is that once I have a model trained, I can use the model to score examples, and then prioritize ones that look more like a bird, so I move towards less buggy and more bird-like images!

Bézier Curves

I also expanded my standard curve drawer to include bézier curves as a first-class citizen. Before, I would just convert them to line segments, but that misses out on some logic, like evenly spacing things along the different types of lines.

I’ve been reluctant for a long time, because I liked constraining myself to circular arcs and lines. But realizing how bézier curves have nice properties (and you can even optimize them for certain properties like in Hobby curves!) I have helpers to make sure the lines, arcs, and curves can match at a tangent (yay for C1 continuity).

Rust Things

Rand

Well, uh, maybe the reason I procrastinate on moving between working in Rust native vs the web is this kind of thing.

Apparently, rand = 0.8 and rand = 0.9 use different getrandom, which have different compile requirements for wasm targets. (I don’t claim to have this all sorted out in my head…). I had dealt with this before when I tried to bump rand, but then reintroduced it when I added a dependency on bvh 0.11.

After some struggle (the ai chatbots going around in circles telling me to upgrade getrandom, then downgrade, then upgrade), I just had to downgrade bvh. The rand 0.8 dependencies go a bit too deep, and I’m not ready to upgrade those yet..

All that said, it is really cool is that my more complicated systems (bird! the wiggly mesh! kaleido!) can easily get packaged for the web.

patch.crates-io

I just learned I can create a global .cargo/config.toml that uses [patch.crates-io] to set the path overrides for all of my local packages. This is so much nicer than setting path on all of my Cargo.toml dependencies!