Today I did something fun. I made a 2.5D game into a 3.3D game.

Some of you know that I have some code for processing 2D video into 3D. I've had sitting in my back pocket for some time intent to see if I could live process a video game with this.

A while ago I did some initial testing by simply processing a short video clip of Doom play through it. Here is what that looks like:

https://gvid.tv/v/BMOdEG

It should be noted that thanks todays work that was more interactive I was able to tune the output further than I had at that point. Also I got my colors uninverted like they are here.

Well it was a success. It has manageable latency, at least for the first level of Doom 2. I took more damage than I'd like for that level. The hallways look pretty real. I was playing brutal doom rather than vanilla so the blood was impressive. That did up the difficulty where you really do need lightning fast reflexes even on the first level. Enemies in the first level are weak but thanks to brutal doom they are pretty aggressive.

Ultimately the way I did it involved screen capturing all of one screen and outputting it full screen in another screen. So really any thing can be 3D. I watched some youtube. That was pretty cool.

If I get latency down this should be pretty practical. From the video you can see that is an anaglyph outout (red and cyan). That's not always the best way to look at 3D. I have been processing video into SBS (side-by-side) output for a while. This can be played on a phone based headset (or ideally a real one)

So the next obvious test is streaming this to my phone over VNC and playing it with a clearer picture but with some increase in latency. I have prior tested such a setup of streaming a 3D desktop environment from my desktop to my phone over VNC. The latency was kind of high. Enough that programming on floating terminal screens was cool and manageable. Even productive. But only just narrowly. I don't think that kind of set up will do well while being shot at.

But this doesn't mean it is ruled out. Steam's streaming is really just VNC they've tuned well, and that is fast enough for gaming over network. So if I tune it right and improve the performance of the post-processing code (not that hard), this might be do-able.

Well that was the most interesting part of today. But there is more I accomplished.

I also made it so the sub-suggestions can't suggest empty subs.

---

I added a supervisor process to the "edge" request system I made for running the bot and other services. The edge-system, while written in NodeJS, is written to be a bit Erlang-ish. So a supervisor process was always meant to be a part of the system. If you don't know what I mean by Erlang-ish, Erlang has a philosophy called let it fail. The don't believe in catching errors. Instead a process should go down and restart. The idea is for the process to have a higher likelihood of operating in a more well known state (less state divergence) and so you are less likely to get logical errors that appear only some of the time under rare-conditions.

The other thing the Erlang community cares about is high reliability. But how can a system with constantly failing processes have high reliability? The key is that the intent that a system is meant to produce can outlive a process. Processes die, requests shouldn't.

Am I there with this system. No. But I'm hoping by adopting Erlang like patterns I can make more reliable systems that can do heavier processing than is typical for a web developer such as me (concurrency, long lived processing). But it's a development style I am new to. So there is if anything an upfront cost to reliability while learning this way of doing things.

Long story short is that Erlang design patterns depend on supervisor processes to restart processes and pipe them restarted work in an intelligent way. My supervisor process is not restarting processes (that's on them to restart), but it is re-initiating work caught in the pipeline if certain criteria are met (timeout period needed to restart an edge, not so old that we consider it failed, not above a recovery attempt count unless code has been updated).

That's the other cool thing about Erlang is hot swapping processes. Requests can half partially fail, you can update code, and then have them complete automatically. All without any downtime when you push the code.

---

Then the last thing I did tech wise today is some follow up from yesterday. Yesterday I helped organize a tech meetup (presented). Basically just did a coding demo of K-Nearest Neighbors and how to speed it up starting from a naive implementation and working our way up.

One thing we did is we used a google form to get people to select their favorite movies and preferred categories (action, drama, skin-flick). Then I made a similarity score between people based on that info. Then for each user I reduced the list down to the top three most similar people to them. Then I found what movies were most recommended by their peers that they didn't themselves select.

So today was just uploading the code to make it publicly available to people in our group and documenting it a little.

---

Oh. I almost missed the first tech thing I did today.

As you know this bot uses a RAG (retrieval augmented generation). Basically it just means that the AI has access to a database of books. Segments of books. Lots and lots of segments. I first was lazy and didn't want to learn new technology so I wrote a c-extension for sqlite3 that let me use it as a vector database. It was good for testing. But then I went ham adding books. I ended up with a query time of over a minute.

Today I finally got around to migrating it to a real vector database, and the query time is down to around 3 seconds. Best part is that won't scale linearly and speeds up as the HNSW graph optimizes (Hierarchical Navigable Small World). Also this can be done in parallel I believe.

So this is one of the reasons why the bot is back. The bot got killed by the document database getting to big for the tech involved. Problem solved.

---

Productive day?