Having a good mapping workflow

[illwieckz]

You know, sometime there is "final release" of map released with missing textures for example, because the mapper had a previous version of his map in his homepath, and the texture loads fine in his config.

Guys who works on assets must be able to test assets on clean path without having to tweak some homepath clone plenty of symlink or to recompile the engine to use different paths. Nobody have to be a sysadmin or a developer to contribute assets.

1. How I work on maps today

I have these 5 directories containing pk3/pk3dir:

Standard paths:

• basepath/pkg

• homepath/pkg

Contributor paths:

• devpath/src

• devpath/build

• devpath/pkg

devpath/src is the directory where I put pk3dir with original lossless assets (tga, wav…), currently I do not use radiant since 95% of the released maps I port do not ship the .map, but if I was a mapper, I will use radiant in this directory.

So, if someone work on radiant, he must be able to work on the map with only these paths:

• basepath/pkg, where are official tex-packages the mapper can use safely

• devpath/src, where is his work, only his work

If someone is an official contributor who works on the official next basepath, he must be able to load only these paths:

• devpath/pkg, where are his tex-packages he wants to test safely (without being fooled by an older one)

• homepath/pkg (or other one), where are maps he wants to test on his tex-package update only

Te release the maps I working on, I use a build tool I wrote that maintain/updates kind of makefiles like that: http://pastebin.com/KvuQ2n5m .

When I run this build tool on devpath/src/map-name_version.pk3dir, it converts the pk3 to devpath/build/map-name_version.pk3dir to a fully functional to-release version:

• convert texture to crn (and sometime png if must be lossless like lightmaps)

• convert sounds to opus

• If I was working on .map files, perhaps the map compilation would take place at that moment too.

• patch bsp to fix sound path since there were converted (see #642 to prevent that).

• build a minimap

• build navmeshes

So, to test releasable maps, I use these paths only:

• basepath/pkg, where are official assets my map can depends safely

• devpath/build, where are the to-be-released files I want to test, and only that

When the map is ok, I use a zipper tool to build the pk3 from devpath/build/map-name_version.pk3dir to devpath/pkg/map-name_version.pk3.

So, that is a workflow I'm able to use today with some hacks:

Code: Select all

# edit assets:
~/devpath/src/map-name_version.pk3dir $ vim scripts/name.shader
~/devpath/src/map-name_version.pk3dir $ gimp textures/name/name.png
~/devpath/src/map-name_version.pk3dir $ audacity sounds/name/name.wav

Code: Select all

# update a kind of makefile with current assets if they changed:
~/devpath/src/map-name_version.pk3dir $ update-buildfile .

Code: Select all

# build a to-be released pk3dir to ~/devpath/build/map-name_version.pk3dir:
~/devpath/build/ $ release-pk3dir ~/devpath/build/map-name_version.pk3dir

Code: Select all

# test map in ~/devpath/build/:
~ $ daemon-testbuild +devmap name

Code: Select all

# build pk3 to ~/devpath/pkg/map-name_version.pk3:
~/devpath/pkg/ $ build-pk3 ~/devpath/build/map-name_version.pk3dir

2. What to do to allow every one to do that

To be able to have the same workflow without being an sysadmin or a developer, we must be able to run daemon without basepath and/or without homepath, or customised pakpath without homepath/pkg while the users's config in users's homepath/config is still used. That task is currently done by a daemon-testbuild hack on my system.

Today to do that we must recompile daemon and/or create tweaked homepath/basepath full of symlinks. Amanieu said +set fs_basepath does not exists anymore, and all must be done with -pakpath, he said too “there is no way of preventing homepath/pkg from being loaded, except by changing the homepath”, so an option to exceptionally not load pkg from default paths is needed. All the workflow described above can be achieved with an hypothetical -nodefaultpaks option.

Code: Select all

# load parpax from homepath/pkg with in-development basepath:
$ daemon -nodefaultpaks -addpaks ~/.Unvanquished/pkg -addpaks ~/dev/build +devmap parpax

Code: Select all

# load in-development map vega with basepath/pkg:
$ daemon -nodefaultpaks -addpaks /var/games/unvanquished/pkg -addpaks ~/dev/build +devmap vega

So, to allow every one to use the same workflow I use, these tasks must be done:

• I must release my build tool or someone write a better one.

• A -nodefaultpaks option must be added to daemon.

• The daemon engine must be able to load sounds whatever the file extension.

Edit:
Bonus: a -neverend option will allow contributors to load unfinished maps or broken maps (missing spawn entities for example), something I can't do today without recompiling daemon with this patch: http://pastebin.com/rWnhi23J . This tweak allowed me to use daemon as a simple map viewer if I play as spectator.

For example with this tweak I was able to render unmodified maps from Urban Terror and other games.

[Amanieu]

With the pk3 DEPS system, it shouldn't be necessary to restrict the pk3s available to the engine. Keep in mind that the engine will only load the pk3 for the current map and its dependencies (and the base unvanquished pk3), but nothing else. The engine will only search loaded pk3s when looking for files, and will never touch a pk3 that hasn't been loaded directly or indirectly through dependencies.

The list of loaded pk3s is shown in the console, and you can check it at any time using /listPaths. The only case where restricting the set of available pk3s would make a difference is when dealing with multiple versions of the same pk3, but in any case you almost always want to use the latest version.

This means that the example you mention here can not occur since we started using the new filesystem:

You know, sometime there is "final release" of map released with missing textures for example, because the mapper had a previous version of his map in his homepath, and the texture loads fine in his config.

The engine will automatically load the map pk3 that has the newest version and will completely ignore older pk3s. Keep in mind that the order of pak search paths is irrelevant to the engine, the only sorting order that matters is the pk3 version number.

[illwieckz]

Yes, the DEPS solve many issue, but it does not solve the problem “I want to have two versions of my map with the same version number at the same time, the one with lossless asset to work on, the one after conversion to test before release”.

[Amanieu]

You can choose which version to use by using -pakpath on the command line. If you wish to keep both src and build directories in your pakpath, then you would have to append a suffix to the version number to make the engine prefer one pk3 to another.