Furblorb

[SyntacticKitsune]

Hi, I may be a bit late (only by around three months), but I've finally gotten this project into a decent state. For those who have somehow not forgotten, this is that project I mentioned in the DeepForest thread that effectively duplicates part of the editor.

So here's Furblorb. It's a command-line tool for turning furballs into projects and back, among a handful of other things. It's written in Java, so it can run wherever Java can (this includes toasters and apparently, debit cards). I originally created it back in December.

Here's a list of Furblorb's capabilities:

• packing and unpacking furballs
• upgrading/downgrading furballs to/from other format versions (note that it doesn't fill in new data for you)
• analyzing furballs -- that is, showing information about a particular furball
• changing the title, author, and dependencies of furballs
• inserting, extracting, and deleting assets of furballs
• theoretically, merging two furballs together (although it won't like, apply scene patches for you)
• shuffling the contents of furballs

That last one is pretty fun actually. It allows one to create a furball where the string tables, item names, etc. have been shuffled. It's reminiscent of Undertale corruptions, for anyone who knows what those are. A warning for the string tables one though: it renders the disposal toggle beyond useless, since disposal scenes can be shuffled such that they display for e.g. the intro text. Anyone who doesn't want to see the disposal stuff should probably stay away from furballs shuffled in this way. It's hard to filter disposal scenes out since the only identifying information is the key -- the only proper way to figure out which texts are for disposal scenes is via some kind of code analysis; anything else would be a heuristic (like checking if the key includes "disposal"). I'd attach a shuffled furball to this post, but it seems the core furball is just barely too big.

Anyway, Furblorb's main capability is basically just doing the editor's job for it. The program will very likely not be useful to those who are already using the editor. And since it only has a command-line interface, it's not terribly easy to use for someone unfamiliar with them. The only real reasons to use Furblorb are:

• If you're not running Windows and can't be bothered to run the editor via Wine (I'm in this group)
• If you want to pack/unpack furballs from the command-line, say for scripting or automation purposes (although theoretically Finmer will eventually be getting its own CLI for this)
• If you want to make use of Furblorb's more bespoke features

Furblorb has been tested extensively -- it can unpack and repack the Core module and get identical results (aside from asset ordering), and it's been used to create DeepForest's furball. It also has some format checks to ensure that e.g. scene patches don't have scene-level scripts.

Furblorb's repository can be found here, and its releases can be found here. It needs Java 17 or newer to run.

---

While I'm here, I'd like to point out a very funny furball deserialization edge-case. I'll just attach a furball to this post since I'd rather not immediately spoil it (see "Broken.furball"). Just load it up (if you have "DeepForest.furball" installed, make sure you move it outside of the modules folder first) and watch Finmer regret reading it in real time. When you're done, here's a description of what's going on: C# doesn't automatically safe-guard against "invalid" (i.e., unnamed) enum constants, so they can leak in if unaccounted for. I specially-crafted this furball to contain undefined compass directions, which Finmer really doesn't like. Basically, Finmer doesn't protect against invalid enum constants in its furball deserialization code (although it does in its json encoder).

Oh yeah, another thing: where did the name "furball" come from? I couldn't find any information on that (I might be blind though). I've speculated that it comes from "tarball", but perhaps there's a different reason (or no specific reason at all).

[Nuntis]

Well hello again! I, uh, wow. Once again you blow me away with how much time you've clearly put into this.

I was actually working on a command-line mode for the editor the other day, so at least some pieces of this will become a built-in tool. It'll have most of the features you listed, sans in-place modifications like editing metadata or assets. So it's an interesting coincidence to see this today. :)

I like the shuffling feature, heh. It's a little bit silly, perhaps, but funny. I'm actually a big fan of randomizers in old games, for instance I sometimes watch A Link to the Past randomizer speedruns, where they shuffle around all the in-game items (so you can, say, find the master sword in a random-ass cave). I wonder if there were more hack-and-slash like gameplay in Core, whether that could be a fun addition.

As for the broken furball: I did give that a try, and what happens is that the scene compiler indeed tries to convert the invalid compass direction to a string, without having an actual enumerator for it, and generates the following Lua code:

Code: Select all

if _CanAppear("Compass_Exit") then
    AddLink(ECompass.15, "Scene_ForestCottage")
end

Since a member-like table access cannot start with a number, ECompass.15 constitutes a syntax error, and script compilation fails. You are correct in your observation that FurballContentReaderBinary does not validate the read constant; this is kind of an artifact of my C++ game engine dev heritage. An enum value validation would require reflection, which is very slow, and I subscribe to the philosophy that if you're hand-modifying furballs (or save data, for that matter), all bets are off anyway :)

Regarding the term 'furball': it's been several years since I picked it (around 2017), so the details are a little foggy, but I'm 95% sure it is indeed a play on tarball + furry.

A question for you, then: was there a particular reason you wanted to use Java? I would certainly welcome some of these ideas as pull requests. ;)

[SyntacticKitsune]

The reason I chose Java was mainly just because that's the programming language I'm most familiar with (although perhaps "familiar" is an understatement). I'm not sure I'd know how to implement stuff like reflective asset ID registration and other similarly intricate parts of the furball specification (or lack thereof) in other languages.

I've seen a few randomizers for other games before -- I think for a small handful of Pokémon games. The reason I pointed out Undertale specifically was because I wasn't sure whether any of these randomizers did stuff like shuffling sprites, sounds, and text. Perhaps at some point I should write a randomizer for items (or even characters -- it'd be pretty funny if the final boss was Iso). There aren't very many builtin randomizers because I pretty much threw all that together on a whim one day.

It's nice to know I was on the right track with "tarball". The name of the program, Furblorb, is also basically just "furry" + "blorb" (since I have messed around with Inform 7 more than a few times). You know, I just realized how it's somewhat ironic that furballs are described as being like zips and yet are not compressed at all (which is something I'm considering addressing, see that last paragraph).

I wasn't aware that validating the enum constants would be that slow. It's probably fine to disregard that suggestion then, since I think I'm the only one hand-crafting furballs. Java's enums don't really have this problem (although arguably they're not enums in the traditional sense either), so I figured validating them in C# would also be relatively cheap.

As for pull requests, I do have some plans for those. In particular there is at least one furball-related thing I want to do which I'm going to make a thread for once I actually have a C# implementation (I already have a Java version). I also have a few feature requests I need to write a post for (and possibly re-evaluate, since it's been three months -- do I really need scripted buffs?).

[Nuntis]

Fair enough! In other programming languages, I guess you could get away with just having big fat mapping tables and lists of classes, to stand in for reflection. Probably annoying to maintain though.

I know ALTTP does have randomizers for a lot of stuff, like doors (any warp trigger), sounds, enemies, items, sprites, and even color palettes what with it being an SNES game. I suppose that could be interesting to play with if Finmer has a little more content to its name.

Regarding furballs being uncompressed - indeed, I've considered compressing them, or perhaps just supporting compressed furballs in addition to uncompressed ones. I suppose it would be a neat addition; I never really got around to it primarily because the gains are somewhat low at the moment. Modules are mostly text, and that doesn't weigh a lot - though admittedly text also compresses really well.

When I say slow, it is relative; it's still on the scale of microseconds of course. C++ brain is just averse to that, heh. But in all fairness, at the time these checks would run, the player is staring at a loading screen anyway, so it's not like I can't spare a couple milliseconds. Something to think about.

Ah, you were thinking of doing PRs and FRs? That's exciting! I'm very interested to see what you're thinking of - no pressure though haha. I haven't forgotten those scene-patch suggestions you made a while back, either.

[SyntacticKitsune]

I initially tried to use a mapping table before I had worked out where the IDs were coming from. It seems like a good idea until you realize just how many visual scripting classes there are (69 of them, apparently -- see "serializables.txt" if you're skeptical). The solution I ended up with was tacking an @RegisterSerializable("<C# class name>") on everything. The bonus to this approach is you can guard against new classes in older furballs just by adding a format version attribute. Definitely better than an Eldritch horror of a mapping table.

For compression, that's something I'll probably talk about more when I get around to making that thread (there are a few other optimizations I think would be worth at least mentioning), but I will say that in my testing GZIP-compressing the core furball does get you a ₆₁.3% decrease in file size (down to ₄₀₀ kiB).

See, I didn't think it'd be seconds slow or anything, but even then it's still additional time spent doing something effectively useless. Part of me is a bit curious now how much time is spent building the initial table of serializable classes, although since it happens once it's probably not a big problem. (I suppose if enum validation was desirable then building an additional table of valid enum values there wouldn't change performance too noticably.) I don't imagine furball loading is too much of a hot spot though -- I mean, most people are going to have maybe three or four furballs at most (which are likely Core + any addons adding more scenes).

As for changes I've thought about, I'll mention this one since I don't plan to work on it for the forseeable future: some kind of text-based interface for playing Finmer, specifically for testing. The idea being you could boot the game and have it run through a text file of commands or whatever and then record a transcript to a file that could then be compared to some known good transcript. I imagine it'd be useful for diagnosing subtle issues should any ever get introduced, since I for one don't plan to play through the entire game after making any changes. Ironically (in the unexpected sense), this might also let the game be playable without needing exciting WINE hacks -- although this is a coincidence, not a reason to seek the change (with how button-driven Finmer is, playing through a terminal sounds pretty bad).

Oh yeah, and one other question: are there any higher resolution copies of Finmer's logo? The maximum size I could find was 32x32 (I think extracted from the ico file). I was wondering mostly because I'd like to have a higher-resolution icon for my furball MIME type. (The current one isn't bad, it's just a little crunchy looking.)

[SyntacticKitsune]

Another question: which subforum would be a good place for the aforementioned furball optimization thread? I thought about putting it in "Feedback & Bugs" although I'm not sure if that's the perfect spot for it. I imagine the other option is putting it in this subforum, although that'd be my third thread in this subforum, which I'm not sure how to feel about.

[Nuntis]

The solution I ended up with was tacking an @RegisterSerializable("<C# class name>") on everything.

Fair; doing all the class discovery through reflection is certainly a lot more flexible, and I agree that attributes (or annotations, in Java lingo) are really nice for making that discovery data-driven.

I don't imagine furball loading is too much of a hot spot though -- I mean, most people are going to have maybe three or four furballs at most (which are likely Core + any addons adding more scenes).

Probably not, no. I'll think about adding validation for that. I'm a bit divided, because it's a bit of a band-aid; you can also change a creature's Strength stat to int32_max and then watch the game try to render two billion dice - and validating a flat integer is much harder. The modus operandi is the same, in that they depend on corrupt modules.

The idea being you could boot the game and have it run through a text file of commands or whatever and then record a transcript to a file that could then be compared to some known good transcript.

That's a cool idea. I suppose it would also need features for controlling combat and randomization (e.g. string sets with multiple entries must have the same one selected, else the test would break). State transitions aren't necessarily deterministic, either, so some constructions would not work. It's possible, but requires some thinking about the edge cases.

Oh yeah, and one other question: are there any higher resolution copies of Finmer's logo?

I've attached the highest-rez I have, it's a Paint.NET file but I also exported a PNG for you.

logo.zip

(17.11 KiB) Downloaded 134 times

Another question: which subforum would be a good place for the aforementioned furball optimization thread?

Feedback & Bugs would be fine for that I think!

[SyntacticKitsune]

you can also change a creature's Strength stat to int32_max and then watch the game try to render two billion dice

I think there is still a difference between this (which results in a hard crash) and rendering two billion dice (which results in a frozen game), although I suppose the end result is similar. I don't think it's a big problem if the enums are left numerical, since as you've said they'd only materialize in rather strange furballs. Anyway, rendering two billion dice sounded like fun, so I just went and did that. I first tried with all 2³¹ stats on the training dummy, as one does, which lead to the game giving the training dummy stats of 0(?). I think what happened was I forgot to subtract 0 so the corresponding number was -1 (or -MAX_VALUE? I don't remember). Anyway, next I tried the more manageable 2¹⁶, which had the exciting effect of completely locking up the game (exactly as expected). Finally, I tried 300 and saw more than a few dice -- so many that they wouldn't all fit in the dice popup. Only after all this did I find out the editor does actually enforce a limit on the stats: 100. Maybe I should add a check to Furblorb...

I suppose it would also need features for controlling combat and randomization

For randomization I imagine being able to "fix" the random with a specific seed would be enough to make it all deterministic. I suppose it would need some easier way of testing other string table entries etc., maybe a runtime flag that Lua code could query (with engine side handling for string tables)? I imagine there's a few options here.

I've attached the highest-rez I have

Thanks for the logo file. It wasn't quite the one I was expecting (I was looking for the one with the black border, like I think the icon of this site is), but I managed to recreate it myself. I've attached a zip with that in several sizes plus an xcf and psd, in case you or anyone else wants them.

Feedback & Bugs would be fine for that I think!

Alright, I've gone ahead and created that thread.

[Nuntis]

Anyway, rendering two billion dice sounded like fun, so I just went and did that.

My poor engine, haha. Yeah, it doesn't quite handle dice rendering intelligently enough to be able to efficiently render hundreds of them; I'm almost surprised 300 works as well as it does then. I expect that most of the time spent rendering 2¹⁶ dice is repeatedly decompressing and instantiating the die template, WPF is notoriously bad at efficiently instantiating controls. The dice themselves, once instantiated, should be simple to render; a GPU can trivially crunch through 65k quads.

It's also why the editor enforces, as you noted, a limit of 100 on the stats. That number is technically speaking completely arbitrary, but I feel like if stats ever go near 100, the module is doing something silly to begin with.

Funny enough, I've tried doing something similar in Armello before, the game that loosely inspired how Finmer's combat system now works. Using Cheat Engine to give myself silly numbers of dice in combat, and seeing how the engine copes. Spoiler: it doesn't really.

For randomization I imagine being able to "fix" the random with a specific seed would be enough to make it all deterministic. I suppose it would need some easier way of testing other string table entries etc., maybe a runtime flag that Lua code could query (with engine side handling for string tables)?

In all likelihood, yeah - all the RNG in the game comes from a single global Random instance; that could conceivably be pre-seeded. I think such a 'playback' feature would be best implemented as a feature on the game executable itself, rather than the CLI, but it's still quite a large chunk of work with a lot of odd edge cases. (How do dialog boxes like shops work? How does user input in combat work? Do those things just need to be bypassed? etc)