Alpha 27 map upgrade (copy)

Ask questions about mapping in general, and show off your in-progress work.
Post Reply
User avatar
Viech
Project Head
Posts: 2139
Joined: Fri Aug 03, 2012 11:50 pm UTC
Location: Berlin

Alpha 27 map upgrade (copy)

Post by Viech »

Here's a copy of the map upgrade instructions given out to our core mappers:

Viech wrote:

Hello fellow mappers! For the upcoming Alpha 27 release (on 2014-05-04), there's a bit of work to do for us. […]

Shared texture packages

They have arrived! Atleast preview versions of them. If you don't hit any issues using those, they will most likely be the first versions to be shipped with the game.

For now there are packages available for the following sets:

  • ex by evillair. This is the main set used on Parpax and Thunder.

  • exm by me. These are additions to ex used on Parpax. (evillair granted us the right to create derivate works.)

  • pk01 & pk02 by philipk. These are used on Plat23, Chasm, Yocto, Parpax, not sure about Thunder.

  • trak5 by TraK. A big set with many color variations that is currently lacking specular maps. Used on crates on Parpax and quite a few other places.

  • space by Ingar. These are the space skyboxes that were posted in this thread.

  • tech by cr4zyb4st4rd. This is the set from this thread in the findings subforum.

[b]Advantages of shared packages and texture compression, skip if you want[/b] wrote:

First of all, as the name suggests, these textures can be shared among the maps, without one map depending on the other. This means that map sizes will be significantly smaller which makes updates less painful for everyone. Obviously this alone will already reduce the total (download) size of an Unvanquished installation. New mappers will also have a quicker start given that they don't need to download and cleanup the individual texture sets or write shaders and readme file entries for them. However, the biggest advantage is in the compression itself. With regard to file size, the compressed packages are much smaller than the zip-compressed source format sets. While the latter are 335 MB total, the compressed sets total up to 109 MB, so literally less than a third. The second big advantage is in the map load time. From what I know, we spent a lot of time, if not most of the time during the map load process, decoding the different texture formats. This part is now completely gone since the new format can be loaded directly onto the GPU. I haven't measured myself but previous tests with compressed base assets showed that the load times were reduced significantly. Pepople have also reported reduced memory usage since we compressed the base assets. Last but not least, I have found that compressed textures unexpectedly look a lot better since the effect of normal mapping is more strongly defined. All in all, I think the upgrade is very much worth the effort.

In order to upgrade your maps, I heavily recommend using Chameleon. The reason is that I tried to use a consistent naming scheme on all the sets. While this will sort the textures in your radiant preview in a meaningful order, this means that you cannot just run sed on your map file to replace the names of the sets. It would also be a PITA to use the very limited texture replacement capabilities of netradiant. The extra effort to install the Python environment and get Chameleon to run will be worthwhile. Make sure to use the latest version of Chameleon with added support for compressed tga, webp and many more formats.

More precisely, what I ask you to do is replace every usage of one of those textures in your map with the textures found in the shared packages. There are different approaches to do so: The first approach is simply using Chameleon to replace the shaders in your map with the corresponding shaders in the shared_* namespace. This should work for most textures. However, in case of lights with custom colors, you will want to ship your own shaders. There are two ways to do so: You can edit your old shaders to point them to the new source files (simple but doesn't produce viable feedback for me), you can extract and modify the shader files inside the packages (recommended; make sure to change the namespace!) or you can use Sloth to generate a whole new shader file from the sources, which I can provide you with when you message me on IRC (they will end up on our upcoming asset host eventually). Unfortunately, sloth cannot currently read .crn files and will fail when used on the compressed texture folder.

Download the latest versions of the preview packages here. You can put them in the pkg directory in the Unvanquished home folder. (In case you received some work in progress version of any of these pk3s before, overwrite them.) Be aware that the new filesystem requires you to explicitly specify dependencies in a file named DEPS in the root of your pk3(dir). For example, if you wanted to use the ex and the pk01 set in addition to the common shaders (which I will write on in the next section), the file would look like this:

Code: Select all

tex-common
tex-ex
tex-pk01

I also added a convenience package that will simply include all other packages, including the common shaders. In your DEPS file, just add the single line:

Code: Select all

tex-all

Before you start retexturing your map, also read the section about gamma renormalization, as you migth want to use lights of higher intensity.

Official™ common shader package

Technically, this is just another texture package but the importance of this set (and the amount of work I had to put in it) made me create a seperate section.

The tex-common_<version>.pk3 package contains both the common.shader file that hosts all the special shaders needed by mappers as well as the corresponding tetxures displayed inside radiant. It does not include any model support files, so buildables will show up incorrectly when you replace Ingar's support package with this. Please do so anyway, so the shaders don't clash. The model support will need to be shipped in a seperate package that shows our new models, ideally together with a fresh entities.def.

I wrote the common.shader file from scratch and it is ridiculous how outdated and wrong the old one was. Many keywords were superfluous (e.g. nomarks is implied by either nodraw or nonsolid, nolightmap is implied by nodraw, etc.) and quite a few of them were simply wrong. There was no difference between nodraw and nodrawnonsolid (both were non-solid) and a few shaders were entirely useless, such as missileclip, noimpact and nolightmap. Here's a list of rules for replacing the old shaders with new ones:

  • clip → Now named playerclip.

  • weapclip, full_clip → Were the same and are now named fullclip.

  • missileclip, nolightmap, noimpact → Removed.

  • nodraw → Is now solid. If you want a non-solid version, use nodrawnonsolid.

  • fullclip → It actually works now but the downside is that radiant will currently not filter it properly. (It previously used the playerclip keyword to lure radiant into treating it as a clip, but this made the shader work like a player clip in return…)

There were quite a few other fixes and changes that I forgot about but shouldn't pose any issues to you (if they do, post about it!). There is also a new shader named black that is basically a solid wall that will always appear pitch black. Should be useful for abysses and cracks. In addition, the shader file is now commented and explains what the different shaders do and can be used for.

Thanks to Ingar for providing the radiant texture sources. The entire content of the package is now in the public domain (more precisely under CC0) so you can use, sample, modify and distribute the content without worrying about anything, not even about attribution.

Please be very considerate that you never ship shaders using the textures/common/ namespace with your map, not even the unmodified ones! While the Daemon engine will not load data from external maps if not explicitly asked for via the DEPS file, radiant and q3map2 could be using your versions of the shaders instead of those that will be shipped with Unvanquished, which can pose hard-to-debug problems to mappers. Ideally, the shaders you find in the package should be sufficient and working for you. If you find any errors or are missing something, I'd appreciate it if you tell me instead of just shipping a custom shader file, so that I can make the modifications available to all mappers.

Gamma renormalization

We have been using a default gamma value of 1.3 so far. This was a very bad decision because obviously mappers didn't want their maps to look washed out and thus adjusted to the new setting, not only making it useless in the first place but also reducing the resolution of the overall brightness distribution as well as the player's clearance for further gamma adjustments (artifacts show up sooner). We aim to correct this with the release of Alpha 27 by resetting it to 1.0 for all clients. In addition, we will also reset r_specularScale to 1.0. These changes mean that all our maps will need to be recompiled with different lights or compile settings or else they would look too dark.

So in short the goal is to make your map look good when r_gamma and r_specularScale are both set to 1.0.

Personally, I found that both passing -scale 2.0 to q3map2 in the lighting phase as well as using shader lights that have twice the old intensity did the trick. Have a look at my experiments with Parpax to see the results. In future maps, you can either add this switch to your default compile options or simply use bigger lamps. Don't use q3map2's -gamma switch, it will screw up the lightgrid. Also please talk to me before using any compile switch apart from -scale so I can test whether this endangers stylistic consistency. Note that increased light intensities can increase map compilation time, but the results are bound to look better than any simple filter applied to the lightmap.

If you don't find the time to complete all the bullet points before Alpha 27, please atleast complete this one so we can introduce the gamma change without your map looking too dark.

Tone mapping optimization

Tone mapping, also known as color grading, is used on most if not all our maps but there are some quirks I'd like to get rid of.

First of all, make sure your tone map is saved in a lossless format. They are very fragile and should not be compressed in a lossy manner. Pevel had to find out that his webp export plugin wasn't doing a lossless export even if the slider was set to 100% quality, leading to color artifacts ingame. If you are unsure, use a different format that doesn't support lossy compression (tga and png should be safe). There are also reports from Mac users that webp images will generally cause tone mapping to fail. I haven't tested the crn format in-depth yet. It's lossy, but maps compressed with it looked fine at a quick glance anyway.

The second point is about what should or should not be achieved with tone mapping. Obviously, they should be used to change the color tone of your map (duh!). I once suggested to Pevel that he should use it to increase contrast, which lead to yocto looking inconsistent with the rest of our map base. I would now advise strongly against changing contrast with tone maps. Please also don't adjust brightness with them. Currently you cannot disable map color grading, but it is not unlikely that users will be allowed to disable this effect in the future. This should, however, not influence how bright the game appears to them.

Reverb

If your map doesn't yet use reverb, add it! There is a wiki article on how to do it and there is a huge set of presets that you can use, potentially with reduced or increased intensity.

Navmeshes

Bot navigation meshes should be shipped within the map from now on. You can find a tutorial on the generation process here. If you need help compiling them for your map, ask me or ds on IRC.

Responsible for: Arch Linux package & torrent distribution, Parpax (map), Chameleon (map texture editor), Sloth (material file generator), gameplay design & programming, artistic direction

Post Reply