Page 1 of 1
Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 4:18 am UTC
by illwieckz
Code_Man sent me an archive of some work-in progress TeraPack dpk, those are free texture sets by Nobiax/Yuqhues.
As a first step, I pushed the files to a repository there: https://github.com/InterstellarOasis/wip-terapack
The purpose of this repository is to be cleaned-up before doing the final repositories. It's a kind of WIP place we can force push if needed.
I did some things;
- converting all TGA to lossless exact WebP, this reduced the source file size from 3G to 1.2G
- denumberizing the test map
- unify some whitespace and things like that in
.shader
files
- make the dpkdir buildable with Urcheon
I decided it is time for us to swtich to WebP as a default lossless exact format instead of PNG, PNG is only good when using optimizers, and using optimizers is slow, and expecting contributors to run optimizers is annoying. When I created the repositories in UnvanquishedAssets I chose PNG as a lossless compressed format because at the time WebP was not yet well-supported, but now WebP is everywhere. So, for newer texture repositories I guess we can safely use lossless exact WebP (lossless exact WebP means we use -lossless -exact
options with the cwebp
utility, because otherwise lossless is not totally lossless).
I have not deleted the .mtr
files we don't use, as some carry some informations I want to be translated to our .shader
files.
Things to do:
- Migrate the file names to our conventions (like
_n
for normal map file, etc.), Urcheon relies on this, and NetRadiant knows how to hide them, this would also allow use to use the Sloth
tool to automatically generate the shader files. Most of it can be scripted.
- 90% of the materials in
.shader
files can be automatically generated with the Sloth
tool, so we better want to switch to that instead of maintaining thousands of lines by hand.
- We may fix some of those materials (I noticed some mistakes like blending a lightmap over a diffusemap, that means blending the lightmap twice), but most of those mistakes would just disappear without having to fix them if we switch to Sloth to generate those materials.
- We may split some normalmap+heightmap files to separate normalmap files and heightmap files as we usually do, this can be scripted, and if we use Sloth to generate the materials, we would just have to name the files properly.
- We may do some folder renaming to fit some conventions, that is mostly scriptable.
The size for produced dpk archives for those files, once crunched, is about 290MB.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 4:35 am UTC
by illwieckz
After examination, it happens that all the materials keywords from .mtr
files we don't support yet (but that we may want to support, sometime it's already partially implemented) are:
Code: Select all
parallaxDepth 0.05
specularExponent 256
specularReflectance 0.4
The values are always the same, so we can safely delete those .mtr
files, the information is now documented here and then, not forgotten.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 5:18 am UTC
by illwieckz
This kind of numbering prevents to generate the materials with Sloth:
Code: Select all
textures/terapack/tree/tree_3
{
qer_editorimage textures/terapack/tree/pattern_116_diffuse
{
map textures/terapack/tree/pattern_116_diffuse
normalMap textures/terapack/tree/pattern_116_normal
specularMap textures/terapack/tree/pattern_116_specular
}
}
After porting to Sloth, it produces:
Code: Select all
textures/terapack/tree/pattern_116
{
qer_editorimage textures/terapack/tree/pattern_116_d
{
diffuseMap textures/terapack/tree/pattern_116_d
normalHeightMap textures/terapack/tree/pattern_116_nh
specularMap textures/terapack/tree/pattern_116_s
}
}
We can rename pattern
to wood
and keep the pattern number, but that means textures/terapack/tree/tree_3
will be renamed to textures/terapack/tree/tree_116
. This probably acceptable, the only thing to rewrite would be the test map. Otherwise I may write a script to renumberzize the images starting with 1, if we don't care to remember the original pattern number.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 5:23 am UTC
by illwieckz
so, I need to know from Code_Man if it is useful or even needed to remember the original pattern number.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 5:28 am UTC
by illwieckz
It may be possible to extend Sloth
to write textures/terapack/tree/tree_3
out of textures/terapack/tree/pattern_116_diffuse
by implementing a name generator based on pattern and iteration in Sloth, but that may be annoying to do.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 8:28 am UTC
by illwieckz
So I did a huge rename of files to follow conventions, and now the materials are all generated by Sloth when building with Urcheon (there is no .shader
file in tex-
dpkdirs anymore).
I have not ported the demo map as I don't know what to do with numbers yet. I have not ported materials frommetalandcomposite-shaderupdate_src.dpkdir
yet.
I noticed that the “metal and composite” package only had metal textures, so I don't know if composite are missing.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 8:34 am UTC
by illwieckz
I noticed that the misc
package is very large, it looks like we can split it in multiple ones, I identified some common themes like floor
and furr
that may be split.
Code: Select all
$ du -shc build/_pakdir/pkg/tex-terapack*
35M build/_pakdir/pkg/tex-terapack-brick-stone_test.dpkdir
24M build/_pakdir/pkg/tex-terapack-grass-plant_test.dpkdir
35M build/_pakdir/pkg/tex-terapack-metal_test.dpkdir
151M build/_pakdir/pkg/tex-terapack-misc_test.dpkdir
57M build/_pakdir/pkg/tex-terapack-rock-sand_test.dpkdir
12K build/_pakdir/pkg/tex-terapack_test.dpkdir
29M build/_pakdir/pkg/tex-terapack-tree-wood_test.dpkdir
329M total
When splitting packages, keeping the original pattern number instead of re-numbering the material from 1
within its destination folder makes things easier, even if the number means nothing from a folder point of view, so I may prefer to keep the original pattern number.
Re: Implementing the TeraPack dpkdirs
Posted: Tue Dec 26, 2023 10:44 am UTC
by illwieckz
So, I ported the demo map to the new texture paths.
It looks like the textures/terapack/road/road_3
path is missing.