MatrixDevFunnyVideosMusicBooksProjectsAncapsTechEconomicsPrivacyGIFSCringeAnarchyFilmPicsThemesIdeas4MatrixAskMatrixHelpTop Subs
4

Typically, for configs, I use two formats. I have JSON and newline-delimited lists. With the newline-delimited lists, I pair them with a live load system to get both a live array and a set from it.

But I've had a new need for a "low structure" format that is accessible to non-tech people. Further along that spectrum is YAML. It has less formatting and is more human-readable than JSON. But it isn't quite there.

Even further in that direction is INI. But INI has a critical weakness. It doesn't support lists. That's actually the main thing I need in the config. But its strengths are twofold. It has sections. This means if INI did support lists, I'd be able to merge a large number of files into one. And it also has key-value pairs. This means I can combine it with what would have been in a JSON config file. With both positive attributes, it means I can have much fewer miscellaneous files for my application.

On top of these benefits, the way I added list support to INI means that lists don't require any decoration or formatting. These beats out almost every known format for simplicity. This keeps with my theme of being good for less technical people, and lists remain as easy as the list files to create.

The trick is kind of obvious. Since INI is a key-value based configuration format, anything that isn't a key-value pair, section header, or comment is a list item.

I also added support for nested sections, which already exist in some modern INI libraries, even though it isn't a required part of the INI standard.


#Global section

banner=https://img.gvid.tv/i/image.jpg
title=My customized forum

[filters.topic]
mode=white
Tech
Funny
Windmills

[filters.origin]
mode=black
jazzforum

It seems kind of obvious. So why isn't this supported in other INI libraries? When we look at the implementation, we will see why. I've implemented it in JavaScript. To do this I have to make use of some unpopular properties of JavaScript that I still think are worth it. If it were written in another language, we'd run into some of the same issues, but we'd have to add some of the same hackiness to those languages since most don't have these hacks out of the box. And thus it would still be hacky in those languages as well, just more novel.

The problem-reaction-solution that gets us to a place some programmers won't like is that we are asking these sections to hold key-value pairs, have list content, and contain each other at the same time. Originally, I was going to assign list content to sectionObj.list, which would have made list a reserved word. The even more verbose option is to separate the three roles. This would mean that with nested sections, we could have things like config.sectionChildren.filters.sectionChildren.users.set.has('user'). Or config.sectionChildren.sidebar.keyPairs.width. Technically this is "more correct," but Yuck!

I prefer: config.filters.users.has('someuser').

The first key thing to notice is that you can use arrays as Objects.

var arr = [1,2,3,4];
arr.width = 2;
arr.height = 2;

JavaScript will also let you use objects as arrays. But using arrays as objects turns out to be more based. It keeps our array content and object content separate. Using an object as an array would coerce our indexes into strings, and the two types of keys are now indistinguishable. We also lose array.length. But if we use Arrays for objects, we get a single object that does both roles seperately.

But I ran into one more imperfection. Adding and accessing the set content still felt clunky. section.set = new Set(section). It was adding a reserved word and another layer of object addressing. Just as before.

So then I found the true best hybrid object in JavaScript. It's the Set. All the advantages I described about Arrays apply to Sets. We get instant lookup with a pleasant syntax. Sets are as iterable as arrays in modern JavaScript. The only thing it can't do that an array can is have duplicates. I'm ok with that.

The other nice thing about sets for configs is that because they describe a membership, they can be used for shorthand rather than just lists.

[domain.com]
route=/var/www/domain
cache
isolate_process
timeout=5000
if(config['domain.com'].has('cache')) {

}

So now the implementation.

module.exports.parseString=parseString;
function parseString(str,base=new Set()) {
 var lines = str.split('\n').map(i=>i.trim()).filter(i=>i).filter(i=>!i.startsWith('#'));
 var section = base;
 for(var line of lines) {
  if(line.startsWith('[') && line.endsWith(']')) {
   console.log('Section start');
   var secName=line.slice(1,-1);
   section = base;
   for(var subName of secName.split('.')) { //Handle nested sections
    section[subName] ||= new Set();
    section = section[subName];
   }
  }
  else if(line.includes('=')) {
   console.log('Setting key');
   var [key,...rest] = line.split('=');
   var value = rest.join('=');
   section[key]=value;
  }
  else {
   console.log('Pushing list value');
   section.add(line); //add() from the Set class.
  }
 }
 //section.set = new Set(section);
 return base;
}

module.exports.readFileSync=readFileSync;
function readFileSync(path) {
 return parseString(require('fs').readFileSync(path,'utf8'));
}

module.exports.readFileProm=readFileProm;
async function readFileProm(path) {
 var fs = await import('fs/promises');
 return parseString(await fs.readFile(path,'utf8'));
}

module.exports.readFile=readFile;
function readFile(path,cb) {
 if(!cb) return readFileProm(path);
 readFileProm(path).then(res=>cb(null,res)).catch(cb);
}


var chokidar = require('chokidar');

module.exports.readFileLive = readFileLive;
//Live updating object
function readFileLive(path) {
 var base = readFileSync(path);
 chokidar.watch(path, { ignoreInitial: true }).on('change', async () => {
  try {
   var fs = await import('fs/promises');
   var content = await fs.readFile(path, 'utf8');
   // Reset the Set-as-object in place, then re-parse into it.
   base.clear();
   for (var key of Object.keys(base)) delete base[key];
   parseString(content, base);
  } catch (error) {
   console.error('Error reloading file:', error);
  }
 });
 return base;
}

Sure, making every object a Set is JavaScript abuse. To me, though, the priority is that the config format makes sense to the end user. If, for the sake of the programmer, we abuse objects, and that isn't well-liked by programmers, that's still no excuse to not offer a reasonable format to an end user.

Hybridizing objects can be cool. Why not make every object a Set? Making every object a dictionary wasn't so bad. In the Java and Ruby worlds, making every primative an object turned out to be good. 5.toString() yo. Why not make files directories and directories files? That's among the more loved features of Plan 9/BeOS/Haiku. Make network interface files. Make hardware peripherals files. Make your CPU temperature a file. Hybridization in some contexts is good. So in this case, every section is a set, a dictionary, and a parent to other sections. If programmers hate it and config authors enjoy it, that's a win.

Comment preview