Visual Novel VM

I’ve been feeling somewhat delusional lately because of this. I am not sure if it is a good idea because (1) it has never been done before; (2) it seems like a misapplication of an interpreter/state machine model; and (3) it takes some work to set up.

In theory, you could turn any sequence of API calls and basic arithmetic into a machine-readable language. Not all encodings are Turing-complete instruction sets, but a number of them are. For instance, I discovered that PostScript is actually a Turing-complete, stack-based language. You can do math in PostScript, define functions, and do things that are far beyond the scope of executing print jobs. If you keep this in mind, you can find that an instruction-based model is not a misapplication for a scripted visual novel. The other question stands, however: is it practical?

The goal behind VNVM is to make visual novels self-containing and portable by adding a layer of abstraction to the behavior of sprites, with the ability for instructions to be sent over the network without any danger of exploitation. While sandboxing is achievable with Lua and Python, I would end up placing an excess amount of emphasis on the security layer to prevent arbitrary code execution, which is ridiculous to defend against using a bytecode that is far more complicated than I really need. Moreover, using Lua in tandem with Python already raises eyebrows: why use two different scripting languages?

A side effect of this project is that you would be able to write visual novels in assembly, if you so choose to. You can also port the VM to the browser, to a GBA, to any platform you can play games on. But the final goal is to make any sort of visual novel play faithfully, without the end user needing to see the backend at all.

Delusional and worried that I was wasting my time, I wondered why the instruction set was so complicated. After taking some inspiration from PostScript, I realized that this approach of a VM, albeit somewhat strange, is workable. I just have to change the instruction set from register-based to stack-based, which was an easy change and eliminated about 100 lines from my code.

My main target right now is Ace Attorney. Currently, case engines are written in exotic languages, such as Delphi, Multimedia Fusion, or some “custom-made” language that looks very similar to AutoIt. VNVM, I admit, is no exception; however, my main argument is that its applications surpass Ace Attorney, and I plan to automate the construction process of a faithful case engine by creating a Python script that emits VNVM machine code. I have already written an assembler; now I am to write a script that uses inline assembly to directly produce machine code.

One useful application of VNVM is writing the behavior of sprites. Users of Attorney Online are accustomed to using .ini files to define emotes, sounds, and delays. But from a technical standpoint, this system is bunk: what if I want my sprite images to be organized in some other way in the file system? What if I need 8-bit transparency in my animations? What if there is something floating around me while I do everything? It is clear that .ini files cannot cover all cases, and even at the introduction of some complexity, the system falls flat on its insufficiency.

My idea was a domain-specific language called Spritelang. This is a macro-based language, which, unfortunately, has rather complicated syntax rules, surpassing my knowledge of compiler theory and the writing of tokenizers. I would certainly write it in JetBrains’ MPS, but I underestimated the notes when they said “this might take about a day’s worth of your time.” The meta-language is not an easy one to understand, but I can sense its immense power currently beyond my reach.

Japan: the hyperfunctional society: part 2

Day 5

I don’t believe there was a wake-up call. I take in the view from the massive windows (supposedly, you can see Mt. Fuji, as remarked by my teacher in one of the earlier travel meetings as an opportunity that she has never been able to have from a hotel room, but it’s cloudy, so it is unfortunately not possible). I will not forget this view. Again, I curse at myself for not bringing my DSLR; the view is too magnificent to be taken by my two cameras, although it is raining and hazy. (more…)

A successful migration

I migrated to the Raspberry Pi successfully. Most of the work was getting the Raspberry Pi back in working order after I accidentally broke the Raspbian installation with an incomplete update that paused right at the libc6 installation two weeks ago.

Most people will tell you that when things start to go awry and you don’t know why, just reinstall everything. The advice, while it may work, is incredibly unhelpful because they fail to understand all the work and reconfiguration that does not get magically copied over on reinstallation. I think it’s just a byproduct of today’s consumerist thinking: don’t bother repairing, just replace. Who cares where the broken stuff goes? The death of shop classes, as I had previously witnessed, is simply further evidence of this consumer culture.

And if you start whining, they’ll tell you to shut up, “managing a server yourself will take many hours,” “read this basic tutorial to the letter,” or “your posts show no research, get out.” And if you really do start making complicated arguments like the absence of fast reconfiguration on reinstall, they’ll tag out and bring in the intellectuals who will evangelize Docker and “write-as-you-go Bash scripts” at your face; “that, or consider paying someone to manage it for you.” What is more insulting than someone telling you outright that acquiring expertise is futile, and to simply give money that you don’t have to an “expert”?

Linux forums depress me, as the “pros” come off as, for lack of a better word, elitist. (There’s definitely a better word to describe them, but I can’t think of it right now.) Basically, if they don’t recognize you or your post count seems fairly low (less than 1,000), they’ll intentionally ask for information about your system beyond what is actually needed.

Anyway, back on topic. In short, I had to reinstall Raspbian, but I first had to back up the partition, which I thought I could do via SSH, but it seems that SSH would randomly halt with “Illegal instruction” around 20% of the way into the transfer. NOOBS doesn’t have SSH or any other file-transfer method (except netcat, maybe), so I just put the SD card in my laptop and transferred the files out. Then, I reinstalled Raspbian.

The configuration process was easier than I had originally conceived. You just install nginx, PHP, and MySQL (which is just a meta-package to install MariaDB – I didn’t know it was a drop-in replacement!), and then you copy in the site’s .conf file to /etc/nginx/sites-available and symlink it to /etc/nginx/sites-enabled. Next, you copy the site data to /srv/[site-name] – don’t copy to /var/www because that’s not the point of /var according to the Linux FHS (but packages aren’t allowed to automatically configure to /srv because the directory’s organization is at the sysadmin’s discretion). Finally, you make www-data take ownership of certain WordPress files, such as everything in wp-content and wp-config.php. To set up the database, just mysqldump from the original server, transfer the query data over to the new server, and pass it as standard input. Don’t forget to re-create the WordPress database account.

Meanwhile, the PowerEdge 2600 is on the chopping block. DIMM bank 1 has been reporting an ECC fault since June, and ever since I restarted the server a few weeks ago, the first drive on the array has been reporting predictive failure, which will bring the array in degraded mode once it fails.

However, why should I put my entire server at the mercy of an SD card that can fail at any time? Hence, I should make a backup configuration now, before it is too late…

Update: I listened to the server spin down for its penultimate time: the PowerEdge has been shut down for good. The RPi has also been set up to perform daily backups of directories with user data and non-default config data on them.

I’ve sunk way too much time into this endeavor, and now I am very behind on my homework. I’ll have to work almost nonstop through the night if I want to get that homework done in time.

On top of that, now I must do a backup of the PowerEdge without arousing questions of what I actually used the server for. It’s almost disgusting how long I convinced myself to use that for, and how much money my dad probably ended up spending over the years just for power. A NAS would have been so much of a greater investment than that pile of junk; now the server is only worth the metal that it is made of.

Migration event soon

I tried to connect to my own website on Friday, but the connection kept timing out. My mind raced with all of these awful thoughts: maybe some script kiddie finally breached the PHP process and decided to bring everything down. Or perhaps a disk failed on the old SCSI RAID array, and now the server is just waiting for me to connect a keyboard and press Enter all the way back at home to start the server in degraded mode.

But alas, the reality was none of it. Upon returning home on Saturday, I entered the attic and saw that the server was off, fans spinning at idle. I impatiently turn it on, the machine roaring to life once again. I supervise the whole process: everything good. Maybe there was a power outage?

Yet more wrong guesses. The culprit was my father, who decided to turn the server off (God knows in what way – did he really push the power button until it turned off?) without any express notice. Later he made an off-hand remark about how he had turned the server off, not knowing that I turned it back on again.

I want – well, now need – to migrate the server. It’s old, it’s heavy, it’s loud, and it’s expensive in power costs (costs about as much as the pool filter in kilowatt-hours per month). It’s pointless to keep it around, and probably embarrassing to explain why I still use it.

My main choices are to throw the site into my Digital Ocean droplet. I could use a Docker container but then I would have to learn how to deal with volatility and general maintenance.

There is also the option to convert everything into Jekyll; the main problem with this is that I am very unfamiliar with Ruby, and I would lose the markup characteristics of HTML (at least that’s the impression they give me). On top of that, I don’t know how to transplant my blog template into a Jekyll template (it’s not my template!) and I don’t want to give into the overused templates they offer. And then after that, where will I host the site? GitHub? There’s no reason for me to push my rants into GitHub, so the world can see what kinds of “contributions” I make every couple of weeks.

Finally, there is the option to move into a Raspberry Pi, which would grant me the benefit of continuing access to my home network, zero maintenance costs (my parents pay for power), and minimal changes to the web stack I currently use.

So immediately before leaving off for college again, at the cost of probably arriving late, I fumbled around for my Raspberry Pi and connected it to the Ethernet port in my room. I guessed the password a couple of times via SSH and then just decided to pull out the keyboard and break into it locally, so that I could remember what the password was. Oh, right, it’s those credentials. I shove the keyboard and dongle back into my duffel bag, gather my other things, and finally set out.

Now, it is my responsibility to get the RPi up to speed, as the new successor of the PowerEdge 2600.

Making an e-bike with display

This is an explanation of another one of those ambitious projects which I really want to do, but I have neither the experience nor the people to actually do it with.

I hate rough inclines: they kill my legs. The number one detractor to riding a bike in my childhood was that in my neighborhood, there are some very steep inclines. It made riding a bicycle not a very pleasant experience, and my father never wanted to bring me to a park for me to ride my bike on, so in the end, I never really used my bike.

However, given the fact that using a bicycle is the only practical mode of rapid transit in the city where I attend college, I want to actually start riding a bike again. And after a year or so of riding that bike, I want to make the riding experience cooler.

First, I want to retrofit a brushless DC motor to the drive shaft; something rated for around 600 W of power output. If it is not possible to attach it directly to the hub, I’ll attach it to the shaft with a belt; ideally, a belt with the quality of a timing belt. But I hope I don’t have to do this, because if so, I’d have to play with the tension, pitch, and so on of the belt, which would be problematic.

Next would be the electronic speed controller and charge controller. I want the controllers to automatically switch to a regenerative mode for slight brakes by bypassing the ESC, inverting the poles of the motor, and taking the current straight to the charge controller. Then, on pedaling, the controllers should switch back to drive mode. This behavior would be directed by the main controller, since regenerative braking is a non-essential feature.

Speaking of a main controller, what exactly is it? The main controller is the Arduino or whatever microcontroller I decide to use that is wired to the ESC and charge controller, but is not required to be run in order to operate the bike in case of a fatal error or low battery charge. It would run a real-time operating system with prioritized continuous tasks and many, many interrupt routines. These would be its high-level tasks, in order of descending priority:

  1. Emergency brake applicator. Continuously checks “emergency stop” button, dead man’s switch (clipped to clothes, but the clamp is limited enough such that it cannot be clipped to the handlebars or other part of bike; then the other end of the clamp is magnetically attached to a port on the control box), or >95% application of brakes while moving at a formidable speed.
  2. 10 Hz alternating pulse. This signal is generated and passes through some kind of failsafe circuit, which then determines whether or not the ESC should be enabled. The alternating pulse ensures that the main controller is not “frozen” on an operation that could prevent it from stopping the motor. This assumes that as long as the pulse is alternating, the controller is working as intended.
  3. Speedometer. It simply samples the speed at which the back wheel is spinning and determines the current speed.
  4. Speed regulator. This task scales back the output DC current based on how close the bike is to the speed limit. This task can be overridden, but it’s not a good idea to do so.
  5. Brake detector. This task detects the brake application percent. The actuation of the brakes is completely analog, but if it is significant, the main controller can signal to go to regenerative mode.
  6. Pedal detector. This task simply detects how much positive force is being applied on the pedal and sets the target DC current proportional to this force (clamped, of course).
  7. Odometer. It uses the same sampling metric as the speed counter, but it increments the distance by the circumference of the wheel. After around .2 miles, it writes to the EEPROM. I suppose I could use a pointer to level the wear on the flash, or I could use a preexisting file system designed specifically for microcontrollers.
  8. Display driver. This assumes that there exists a layer of abstraction between the UI and the display itself.
  9. Sound driver. Just for basic beeps and boops.
  10. Main UI. This handles button interrupts (the calls of which are passed to the foreground user task), the failsafe UI (if all user-mode UI tasks are dead), and the UI toolkit itself.
  11. Foreground user task. Dashboard, options, etc. Must not directly control motor operation.
  12. Background user tasks. Battery icon, clock, etc. Must be non-critical.

The e-bike’s main controller would require a key for operation and then a simple on/off SPST switch located in front of the handlebars. The display would ideally be a Hitachi HD44780-esque LCD, but it could also be the Nokia-style LCDs, although these might be a little too small. There will be six buttons: on the left below the display, there will be four directional buttons laid horizontally (in a style familiar to Vim users or Dance Dance Revolution/StepMania players), and on the right, a back button and an enter button. The display and controls need to be water-proofed.

Instead of using heavy deep-cycle lead-acid batteries, I’d just opt for using LiPo cells, which are ubiquitous in hobby usage for high-performance electronics. Industry professionals are not fond of LiPo cells because they are comparatively more dangerous and volatile than other types of cells, and this increased risk cannot be tolerated in mass production. However, since I am not mass-producing e-bikes, it should be OK to accept the risks and enjoy the power of lightweight LiPos, as long as their charging is supervised closely.

This e-bike also needs a brake light, signal lights, and an LED headlight with a white color temperature rather than blue.

That’s all I want the bike to do. All of this, but I want to keep it street-legal and be able to prove that it can be safely ridden in busy streets under the consideration of various fail-safe mechanisms, including a speed regulator that requires manual override.

Sadly, I don’t know if I will ever be able to make this contraption.

Where’s the good backup software?

For *nix users, the answer is easy: rsync. For Macintosh users, the answer is even simpler: Time Machine (“time ‘sheen”). For Windows, the answer is a convoluted mess of choices. And the problem is that none of those choices give everything you want.

Why can’t you have everything? Here’s all of the things a backup program needs:

  • Permissions. If you can’t preserve your metadata, forget about making faithful backups. POSIX and Windows permissions are very different, but they still deserve the same love.
  • Resilience. The restore part of a program should never produce a fatal error, unless a backup has been corrupted beyond repair. If a part has been corrupted, ignore that part, notify the user that a corrupted portion was ignored (noting, of course, what the corrupted portion actually is), and continue with the restore process.
  • Compression. Many would argue that compression only makes the backup more difficult to restore, yields a minimal return in efficiency, etc. However, this can make a large difference when uploading from a personal home network to a storage service, where storage costs are billed by the gigabyte. I don’t know about you, but $1 a month was more than my tax return this year.
  • Encryption. Everyone’s got their tinfoil hats on, how about you?
  • Incremental backups. People are not going to do full backups every week. This is a waste of time, storage space, and energy, since most files would be redundantly stored.
  • Block-level. If you modified a 20 GB VHD file, are you going to copy that whole thing on every weekly incremental backup? No, you’re going to copy the differences in blocks/parts of that file.
  • Archivable. It appears most people choose either image-based backups or file-based backups. I personally prefer at the file level, but this should not mean “copy millions of files and spew them on the target directory.” The backup should be neatly organized in, say, 50 MB parts that can be easily uploaded to a cloud service as part of a future backup plan. Or, it can just be made as a monolithic 800 GB file. The former is workable by most consumer file services, while the latter is most convenient for more enterprise-oriented services like Amazon Glacier.
  • Resumable. Most backup programs hate it when you shut down your computer for the night. Yet none of them seem to understand that this is exactly what shadow copies are for. Even after shutting down the computer, shadow copies do not magically change. Yet the software goes, restarts your entire backup, and creates yet another useless shadow copy for the mere sake of not wanting to touch files in use and making the most up-to-date backup possible.
  • Snapshots. Let’s say I don’t want to restore my whole computer; I just want to see an old file and its version changes over time. Most backup programs will not let you do that, citing that it is “too complex.” No, it’s not. Track the files the software backed up, using a tiny database like SQLite. There, you can store checksums, file sizes, previous versions, and so on and so forth. The suffering ends there. The end user can view a snapshot of the computer at a certain point in time, or view the history of a specific file, perhaps with diffs (binary diffs if the backup software is user-friendly enough).
  • Low profile. What is CloudBerry Backup using 2.7 GB of memory for? Just flapping around? No! Decent backup software should use 100 MB of memory, tops. Leave the heavy RAM consumption to browsers, games, and servers.
  • Integration. This backup software should be robust enough to make anything either a source or a destination for backups, notwithstanding the limitations of each backup medium.
    • Least liquid: Offline local storage; Amazon Glacier; Google Coldline
    • Somewhat liquid: FTP (due to its slow transfer speed of many files and inability to perform multipart transfers); most consumer storage services
    • Most liquid: iSCSI SANs; high-availability storage services
  • Drive path-agnostic. A backup software should never, ever depend on drive letters to figure out backup sources and targets.
  • Predict drive failure. This goes somewhat beyond the scope of a backup software, but there should be at least some kind of periodic SMART monitor to inform and warn a user of a drive that is indicating signs of failure. Yes, put a big popup on the notification bar with a scary message like “Your drive might fail soon” or just outright “Your drive is failing.” Show it to them the first three days, make it go away, and then show it to them the next week. Of course, the notification can be removed for a specific drive, but it will require them to read a message about possibly losing data on the failing drive, wait 5 seconds to close the dialog, and now they never have to see the dialog for that drive again.
  • Recognize cache folders. Here’s what you need to do: just stick that CCleaner scanning stuff into your product. Make the default backup plan ignore whatever CCleaner would usually clean up. Caches can add up to be gigabytes of size, and many users do not even care about including them in their backups, because all they want are their programs and documents. However, there is that one company that might say, “no you can’t ignore cache folders because we need a perfect file-level backup of the system tree.” (My argument would be to use CloneZilla and do it at the image level – but fine.)
  • Import from other services. No, I don’t care much about Acronis, Veeam, or other proprietary solutions. What I do care about, however, are the crappy Windows 7 Backup and Restore backups, dd “backups,” and other image-level backup formats. Don’t just import the backups: import file history, recompress them, preserve timestamps. Give them the full treatment, and put them neatly in the new backup format as if it really were an old backup.
  • Responsive (and responsible) backend. Big enterprise backup software uses a UI frontend, which merely communicates with the service backend. This is generally a good design. However, when the backend decides to quit, the UI frontend goes into limbo and does not respond to any commands, instead of providing a reasonable explanation to what is happening with the backend, while the backend does not attempt to halt whatever blocking operation that is taking too long. The gears just grind to a halt, and nothing can get done on either side.
  • Don’t delete anything without asking. No, I don’t even want an auto-purge functionality, and if you do, for the love of God, make it a manual operation. There is no reason to keep purging things constantly, unless you have a disk quota to work under – in that case, the software should determine what is best to purge (start with the big stuff, at the earliest backup) to meet the size requirement.
  • Only one backup mode. That backup mode better be good, and it should have a hybrid format.
  • Open-source format. The software itself may not be open-source, but you are essentially ensuring that someone out there can make a restore software that can always be compatible with the latest and greatest operating systems.
  • Bootable. Where are you going to make your restores from? A flash drive running Linux with an ncurses interface for your backup software, obviously. You could, of course, allow backups from that same bootable drive, in the case of an infected drive or as part of a standard computer emergency response procedure – but eh, that’s really pushing it. Just restores will do fine.
  • Self-testable. Make sure the backups can actually restore to something.
  • Exportable. One day, your backup software will not be relevant anymore, so why bother locking in users to your format? Make it so that they can export full archives of their backups, with a CSV sheet explaining all of the contents of each archive.

At the end of the day, users just want their files safe and sound, so keep the software as close to the fundamentals as possible, and allow others to make tools around the backup software if additional functionality is needed.

Paranoia about the eclipse

Here it is in TL;DR format:

  • If you didn’t spend $500 on the latest ISO for this exact solar eclipse event, don’t use equipment thinking that it “blocks the dangerous solar rays.”
  • When the Moon passes over the Sun, the Sun becomes an ultra-hot beam of plasma ready to annihilate anything that it touches.
  • You are an idiot because you are a non-professional who tried to look at the Sun.
  • Don’t look at the Sun or your eyes instantly bulge out of your eyesockets and explode.
  • $100 for eclipse glasses? Well, it’s only for a few minutes, and they make looking at the sun safe, so I think they’re worth the price ;)))))
  • Stay indoors because the zombies are coming.

When I was a kid, I used to look at the Sun for a second or so at a time. Did it make me a better person? No, but my vision was unaffected: I still do not wear glasses to this day. I can’t say the same thing about these days. My eyes have become older, and when I do look at the Sun, it forms spots on my eyes where the Sun was, and the spots linger for a few minutes until they consume themselves.

If you want real advice, go here: http://web.williams.edu/Astronomy/IAU_eclipses/look_eclipse.html

Ideas for a new operating system

As I was watching Druaga compare the Windows system folder with the Mac system folder (which is probably just a fake frontend to a really ugly backend), I suddenly began to pace around, thinking about that graph-based file system again. I also thought about the “organization” defined by POSIX: is /usr/, /var/, /etc/, /opt/, /lib/, etc. really understandable? There’s clearly a conflict here: we want an organization that caters to both readability by the user, the core system, its components, and applications.

I speculate the creation of a new operating system in the next generation. Currently, I believe that it is nearly impossible for a new kernel to be created due to the excess complexity that semiconductor companies have thrown into their electronics, rendering operating system support for PCs effectively exclusive to Linux and Windows since those are the only two systems that they really test.

Why do we need a new operating system? In truth, we really do not. The conservatives will say, “Then why go into so much effort making an operating system that nobody will use, when there already exists one that works?” I’d ask the same thing about GNU Hurd, ReactOS, and so on.

It’s for the future. You see, there is a fatal flaw in the current operating system organizational architecture: it’s a limited directed graph that surmounts to a tree. It works under the premises that system data can be organized as papers and files inside folders. But the truth is that such data can be organized in various ways, but still not necessarily in a hierarchical or tag-based structure.

An undirected graph-based file system would work like the brain, and using more fundamental pieces of data that could allow the cluster size to go down to perhaps 2K. It would be incredibly difficult to visualize, but what you could still do is place sections of this data system in different physical locations, such as a server.

(more…)

A visit to the Googleplex

After doing a thing with Google for the summer with a team of college, 150 or so of us were given an all-paid-for trip to the Google main headquarters in Mountain View, CA, for having completed the primary goals of the coding project.

It is certain that there are a very few number of individuals that get this opportunity. If you were just a kid, you’d be jumping up and down, but we are mature individuals (and broke college students) and know better than to get our hopes too high.

Because we were not informed at all that we were forbidden from disclosing any part of the trip, I can make full disclosure – well, at least the most interesting parts.

(more…)

Japan: the hyperfunctional society: part 1

This is intended to be a complete account of my events in an eight-day trip to Japan, which had been planned for about two years by my native-speaking Japanese teacher, was organized by an educational travel agency, and included 26 other Japanese students with varying levels of knowledge.

Names have been truncated or removed for the sake of privacy.

After many intermittent lapses in editing, I decided to just split it into two as it was getting increasingly difficult to get myself to finish the narrative, but at the same time did not want to hold back the finished parts. I am not intending to publish this for money or anything like that; please excuse my limited vocabulary and prose during some dull parts. (more…)