Why we need a real raytracer for lightmaps rendering

[chris]

Yes its definitely possible. I already rendered external lightmaps in giles and modo.

However it takes ages until a scene converges with global illumination on.
Especially with just cpu+unbiased renderer.

UNV with modo lightmap GI+softshadows, iirc I stopped there after 20 min progressive rendering.

giles lightmap, hard shadows

Q3map2

If q3map2 would use some dithering it would look much better btw.

There is a thing I thought about, q3map2 knows how to read and render .shader files. It's its job to do it to render lightmaps, since it's just painting shadow and lights over .shaders it computed and rendered first. So, is it possible, for every <shadername> referenced by a bsp and found in a .shader file, to export to an arbitrary directory a tga file that contain the computed texture from <shadername>. This way, blender (or any other renderer) can load the precomputed shader as textures. What do you think about it? Is it possible to modify q3map2 to export a computed tga for each shader?

No idea what you mean, but that is not needed for lightmaps. All you need is already saved in the final bsp except light position.
(but I believe there is a flag to keep lights?).

[Viech]

No idea what you mean, but that is not needed for lightmaps. All you need is already saved in the final bsp except light position.
(but I believe there is a flag to keep lights?).

As mentioned earlier, please keep in mind that no serious map uses actual light entities as they have always produced really bad results (either light without a source or light souces that are lit more brightly than the area they are supposed to light). I can imagine that q3map2 internally generates light entities for light emitting shaders (which are used by the hundreds in every map), too, but it may be that a whole collection of directed and undirected lights are generated like this per single light source. Hence it is possible that the amount of "internal light entities" that q3map2 considers goes into the thousands.

[chris]

No idea what you mean, but that is not needed for lightmaps. All you need is already saved in the final bsp except light position.
(but I believe there is a flag to keep lights?).

As mentioned earlier, please keep in mind that no serious map uses actual light entities as they have always produced really bad results (either light without a source or light souces that are lit more brightly than the area they are supposed to light).

All of our experience with q3map2 lightmaps does not count in this case, with a modern path tracer you don't have this flaws from q3map2.
It's a complete new workflow, that we have to figure out first what works best for our maps.

It's also not limited to just point lights you can harness all the light types blender or whatever renderer is capable of.

[illwieckz]

As mentioned earlier, please keep in mind that no serious map uses actual light entities as they have always produced really bad results (either light without a source or light souces that are lit more brightly than the area they are supposed to light). I can imagine that q3map2 internally generates light entities for light emitting shaders (which are used by the hundreds in every map), too, but it may be that a whole collection of directed and undirected lights are generated like this per single light source. Hence it is possible that the amount of "internal light entities" that q3map2 considers goes into the thousands.

It looks like q3map2 just generate multiple pointlights in front of "light emitting shaders", you say how many the surface must be subdivided (which computes how many pointlights for a same surface) and the distance of these pointlights. If you don't subdivise it a lot, you just get some big globs (you can see them on my metro screenshot above, even with light emitting shaders). This is why for example you get all these wrongly casted shadows since the shadows are casted from all these pointlights from subdivisions.

Btw, this is some work I did with some conversion (q3map2 bsp to ase, assimp ase to 3ds):

faces (automatic conversion)

fullbright (automatic conversion)

fast lighting (light emitting shaders remade by hand, just to see how it works)

q3map2 can export bsp to obj or ase. Blender can import obj without external plugins, assimp can convert ase to some formats Blender can import without external plugins like 3ds, ade (collada). I strongly hope we can use that ade interchange format for blender import since it's the fastest I tried (there is also a dedicated library for that task, that's probably why it's fast), badly, the current ade imports references textures but something goes wrong (they are full transparent and without diffuse), probably something to fix somewhere in assimp.

q3map2 can export bsp to obj or ase but do not export curved mesh, that is very bad, perhaps it's a limitation of obj? I don't know for ase but these curved meshes are dropped too. This must be fixed.

q3map2 knows how to convert q3 shaders in mtl or internal ase format, but only does it for diffuseMap textures, it must be extended to export specular, normal and other special textures. It must also be extended to convert light emitting shader informations (we can think of adding new keywords that target the final renderer if the final renderer handles stuff q3map2 don't do).

This is how I load a map in blender:

first, convert all textures to tga:

Code: Select all

find $(find * -maxdepth 0 -type d | grep -v build) -name '*.jpg' -o -name '*.png' -o -name '*.webp' | while read j; do t=$(echo $j | sed -e 's/\.jpg$/.tga/;s/\.png/.tga/;s/\.webp/.tga/'); convert $j $t; done

then compile the map:

Code: Select all

q3map2 -game unvanquished -fs_pakpath /data/unvanquished/pkg/ -fs_pakpath . -bsp -meta -custinfoparms -samplesize 16 maps/rsmse.map
q3map2 -game unvanquished -vis -saveprt maps/rsmse.bsp

then convert to obj+mtl:

Code: Select all

q3map2 -game unvanquished -fs_pakpath /data/unvanquished/pkg/ -fs_pakpath . -convert -format obj maps/rsmse.bsp

to convert to dae or 3ds I do:

Code: Select all

q3map2 -game unvanquished -fs_pakpath /data/unvanquished/pkg/ -fs_pakpath . -convert -format ase maps/rsmse.bsp
assimp export maps/rsmse.ase maps/rsmse.dae -fcollada
assimp export maps/rsmse.ase maps/rsmse.3ds -f3ds

[Viech]

Yes, as Illwieckz just outlined, my point is that q3map2 reads .shader files to spawn lights in front of relevant surfaces. These lights are not in the .map file and not in the .bsp file either! These q3map2-spawned lights are those that are generally used while light entities are mostly used by unexperienced mappers or in non-release-quality maps. It is therefor not sufficient to have a renderer that works only with entity lights you can find in the .map (and preserve within the .bsp).

[illwieckz]

rsmse map in blender (not so many curves, so almost all faces are there, but not all):

metro map in blender (many curves, so almost everything is missing):

[illwieckz]

my point is that q3map2 reads .shader files to spawn lights in front of relevant surfaces. These lights are not in the .map file and not in the .bsp file either!

since q3map2 in -convert mode read shaders to generate .mtl (or similar), I think it could be possible to extend q3map2 to read those q3map_ calls to export them to real emission keywords in some of the exported format (mtl or builtin ase).

[chris]

Yes, as Illwieckz just outlined, my point is that q3map2 reads .shader files to spawn lights in front of relevant surfaces. These lights are not in the .map file and not in the .bsp file either!

Every face in the bsp file is attached to a shader/texture.
A naming convention is everything you need to find these particular faces and turn them to light emitting surfaces in the path tracer.

E.g how you already did in parpax "squarelamp_orange_10k".

[illwieckz]

I wrote an issue on netradiant's issue tracker: #79: Exporting patch meshes (curved meshes) on -convert stage. As the title says, it's about getting the whole map model converted. Textures and lighting issues can wait if we don't have a complete model export.

Every face in the bsp file is attached to a shader/texture.
A naming convention is everything you need to find these particular faces and turn them to light emitting surfaces in the path tracer.
E.g how you already did in parpax "squarelamp_orange_10k".

There is more advanced stuff in q3map2 than just emitting light, like "surfaceparm lightfilter" for example, see tipsq3map2.html#vitrail. Beware, French language inside, so I put the screenshot there, it's better than a lot of words:

.

[illwieckz]

That is a map to bsp (q3map2) to ase (q3map2) to 3ds (assimp) rendered in blender/cycles.

without textures:

with textures:

The light emission shader were remade, the toolchain must be improved to not have to remake them of course. It's not a lightmap rendering, it's just a scene render with diffuse maps and some light emissions.

Do not mind the badly placed models at the center of the map (probably an issue with trains while exporting).

By the way, there is a big issue with this toolchain, the loading final file in blender (loading ade or 3ds converted with assimp from ase, or obj from q3map2 directly) is very slow, take multiple hundreds of second, and the rendering is as slow.

When I convert a bsp to dae directly with assimp, the assimp conversion does not last more than 2s and the blender loading is instantaneous. In fact the bsp map compilation is the slower things in the process (20s for the metro map). So perhaps it's better to improve assimp than improving q3map2. And perhaps, one day, add a map importer direclty in assimp?

By the way, these are the issues I found:

q3map2 can't export patch meshes to ase/obj, I think it's a format issue for obj, probably the same for ase.
assimp can't read bsp if they are not embedded in pk3 first (see the #1030 assimp issue: “q3bsp importer is a pk3 importer, not a bsp importer”).
assimp does not export well diffuse texturing from bsp (but does it well from 3ds but bsp-ase-3ds is a very inefficient process).
whatever the toolchain, the light emission shader must be converted.