Implementing the TeraPack dpkdirs

[illwieckz]

Code_Man sent me an archive of some work-in progress TeraPack dpk, those are free texture sets by Nobiax/Yuqhues.

As a first step, I pushed the files to a repository there: https://github.com/InterstellarOasis/wip-terapack

The purpose of this repository is to be cleaned-up before doing the final repositories. It's a kind of WIP place we can force push if needed.

I did some things;

• converting all TGA to lossless exact WebP, this reduced the source file size from 3G to 1.2G
• denumberizing the test map
• unify some whitespace and things like that in .shader files
• make the dpkdir buildable with Urcheon

I decided it is time for us to swtich to WebP as a default lossless exact format instead of PNG, PNG is only good when using optimizers, and using optimizers is slow, and expecting contributors to run optimizers is annoying. When I created the repositories in UnvanquishedAssets I chose PNG as a lossless compressed format because at the time WebP was not yet well-supported, but now WebP is everywhere. So, for newer texture repositories I guess we can safely use lossless exact WebP (lossless exact WebP means we use -lossless -exact options with the cwebp utility, because otherwise lossless is not totally lossless).

I have not deleted the .mtr files we don't use, as some carry some informations I want to be translated to our .shader files.

Things to do:

• Migrate the file names to our conventions (like _n for normal map file, etc.), Urcheon relies on this, and NetRadiant knows how to hide them, this would also allow use to use the Sloth tool to automatically generate the shader files. Most of it can be scripted.
• 90% of the materials in .shader files can be automatically generated with the Sloth tool, so we better want to switch to that instead of maintaining thousands of lines by hand.
• We may fix some of those materials (I noticed some mistakes like blending a lightmap over a diffusemap, that means blending the lightmap twice), but most of those mistakes would just disappear without having to fix them if we switch to Sloth to generate those materials.
• We may split some normalmap+heightmap files to separate normalmap files and heightmap files as we usually do, this can be scripted, and if we use Sloth to generate the materials, we would just have to name the files properly.
• We may do some folder renaming to fit some conventions, that is mostly scriptable.

The size for produced dpk archives for those files, once crunched, is about 290MB.

[illwieckz]

After examination, it happens that all the materials keywords from .mtr files we don't support yet (but that we may want to support, sometime it's already partially implemented) are:

Code: Select all

parallaxDepth 0.05
specularExponent 256
specularReflectance 0.4

The values are always the same, so we can safely delete those .mtr files, the information is now documented here and then, not forgotten.

[illwieckz]

This kind of numbering prevents to generate the materials with Sloth:

Code: Select all

textures/terapack/tree/tree_3
{
	qer_editorimage textures/terapack/tree/pattern_116_diffuse
	{
		map textures/terapack/tree/pattern_116_diffuse
		normalMap textures/terapack/tree/pattern_116_normal
		specularMap textures/terapack/tree/pattern_116_specular
	}
}

After porting to Sloth, it produces:

Code: Select all

textures/terapack/tree/pattern_116
{
	qer_editorimage textures/terapack/tree/pattern_116_d
	{
		diffuseMap textures/terapack/tree/pattern_116_d
		normalHeightMap textures/terapack/tree/pattern_116_nh
		specularMap textures/terapack/tree/pattern_116_s
	}
}

We can rename pattern to wood and keep the pattern number, but that means textures/terapack/tree/tree_3 will be renamed to textures/terapack/tree/tree_116. This probably acceptable, the only thing to rewrite would be the test map. Otherwise I may write a script to renumberzize the images starting with 1, if we don't care to remember the original pattern number.

[illwieckz]

so, I need to know from Code_Man if it is useful or even needed to remember the original pattern number.

[illwieckz]

It may be possible to extend Sloth to write textures/terapack/tree/tree_3 out of textures/terapack/tree/pattern_116_diffuse by implementing a name generator based on pattern and iteration in Sloth, but that may be annoying to do.

[illwieckz]

So I did a huge rename of files to follow conventions, and now the materials are all generated by Sloth when building with Urcheon (there is no .shader file in tex- dpkdirs anymore).

I have not ported the demo map as I don't know what to do with numbers yet. I have not ported materials frommetalandcomposite-shaderupdate_src.dpkdir yet.

I noticed that the “metal and composite” package only had metal textures, so I don't know if composite are missing.

[illwieckz]

I noticed that the misc package is very large, it looks like we can split it in multiple ones, I identified some common themes like floor and furr that may be split.

Code: Select all

$ du -shc build/_pakdir/pkg/tex-terapack*
35M	build/_pakdir/pkg/tex-terapack-brick-stone_test.dpkdir
24M	build/_pakdir/pkg/tex-terapack-grass-plant_test.dpkdir
35M	build/_pakdir/pkg/tex-terapack-metal_test.dpkdir
151M	build/_pakdir/pkg/tex-terapack-misc_test.dpkdir
57M	build/_pakdir/pkg/tex-terapack-rock-sand_test.dpkdir
12K	build/_pakdir/pkg/tex-terapack_test.dpkdir
29M	build/_pakdir/pkg/tex-terapack-tree-wood_test.dpkdir
329M	total

When splitting packages, keeping the original pattern number instead of re-numbering the material from 1 within its destination folder makes things easier, even if the number means nothing from a folder point of view, so I may prefer to keep the original pattern number.

[illwieckz]

So, I ported the demo map to the new texture paths.

It looks like the textures/terapack/road/road_3 path is missing.