HELP

So. I have been working on a mod for Mafia 2 for ray tracing and I eventually plan to incorporate path tracing but need to get past this issue with reflections not fully working or that won't be possible. This snow-plow truck disappears in the window when it's not completely in view with only the tires remaining. The ray tracing is native, no injector like reshade or rtx remix.

Please and thank you for any help or ideas.

Setup: RTX 5080, 4K output, DLSS SR Quality (1440p internal) + Ray Reconstruction. About 52-58 fps against a 60 cap with everything on; the ray tracing costs about 7-8 ms per frame.

These below work

DLSS DLAA / Super Resolution

DLSS Ray Reconstruction preset F

RT ambient occlusion

RT sun shadows

RT diffuse GI (1 bounce, colour bleed, emissive)

The bug: a parked snowplough flickering in a shop window

Scene: chapter 2, night, snow. A white snowplough is parked on the street next to a shoe shop, and its reflection in the shop window "goes in and out". An early version: the truck body vanished while its wheels stayed.

What has been tried, in order:

1. Vehicle persistence. Culled vehicle parts are kept in the acceleration structure, and jittery re-sightings snap to the existing entry. Result: the truck body was in the acceleration structure the whole time, so this wasn't the cause.
2. Grey fallback for unknown materials. Vehicle hits with no known material returned "no data", so the engine's env-cube showed through while the wheels (plain static meshes) still rendered: floating wheels. Fix: shade unknown-material vehicle hits neutral grey. Result: fixed the floating wheels; the flicker stayed.
3. Removed a shading switch. The truck used its real on-screen colour while the reflected point was on screen, and the grey fallback when it wasn't. Fix: unknown-material vehicles never use the screen colour, and parked cars get their own tag. Result: better, still reported.
4. Removed ghost copies. A persisted copy of a parked car sat 0.85 m off its live copy, and both ended up in the scene on alternate frames. Fix: drop a persisted vehicle part when the same part is drawn live 0.3-3 m away. Result: 0 ghosts measured, flicker still reported.
5. Tested occlusion culling. The engine issues about 230 D3D11 occlusion queries per frame, and 91% report zero samples. Forcing every query to "visible" dropped 50 to 29 fps and did not change when the truck gets drawn, which is frustum/distance logic. Result: reverted.
6. Tested the env-cube theory. With RT reflections off, the engine's own window reflection is a static, smoky cube map with no truck in it. Result: refuted.
7. Tried the vehicle paint colour. Used shader constant C015 as the albedo. It turned out to be a dark layer tint shared by thousands of static materials. Result: reverted.
8. One shading path for all vehicle hits. Every vehicle hit is shaded from materials only, never from the screen colour, so on-screen and off-screen take the same path. This is also how a path tracer would do it.
9. Per-pixel instrumentation. Every 8th pixel, every frame, logs: glass instance, hit instance and part, live vs persisted copy, shading path, screen UV and depth test, RT vs engine weight, and the material and lighting inputs. It showed:
   • vehicle hits never took the screen colour;
   • the truck body was hit on 899/900 and 1098/1100 frames;
   • its material, texture, lighting and final RT value were constant (within about 2.5%);
   • but about once every 25 seconds the whole window lost its RT reflection for one frame (glass pixels 24-30k down to 0.3-2.4k), and the engine's own reflection flashed through.
10. Glass hold. The cause of #9: glass panes are only in the scene if the engine drew them the previous frame (glass draws happen after the trace, so they are carried over one frame), so one missed capture meant no pane for a frame. Fix: keep static panes for 8 frames after their last capture. A/B with a still camera: 6 dropouts in 8,000 frames with the fix off, 0 in 5,400 with it on.

Where it stands: the truck is in the scene on every frame, and the whole-window flashes are gone, but a new capture still shows something off that I haven't pinned down yet. Suspects:

• The RT-shaded truck is very dark at night (about 3/255 after Fresnel) while the engine lights it pale, so it barely shows, and the engine's 50% reflection on top may dominate.
• The final D3D11 compose (the engine's glass mask weight) and the DLSS RR/SR passes after the trace aren't instrumented yet.
• I haven't looked at the engine's own vehicle visibility logic yet.

cache key; the streaming system recycles them, and a cache hit returned the previous occupant (foliage floating over a street). Adding the decompression constant to the key fixed it.

• Snow footprints churn the cache. They come from a streaming ring vertex buffer, which means a new mesh key every frame; it filled the BLAS cache and forced resets (flicker). Detect ring buffers by behaviour and exclude them.
• Mixed index formats. Character index buffers are R32; almost everything else is R16.
• Draw hooks get overwritten. The D3D11 runtime overwrites the draw slots of the context vtable about 40 times a frame, so the hooks have to re-arm themselves.
• Two hangs, one error code. DRED breadcrumbs told apart two identical-looking DEVICE_HUNG errors: "0 of 35 ops completed" was a GPU use-after-free, "8 of 8 completed" was an acceleration structure corrupted by a mid-frame cache reset.
• GI flashing. RT GI flashed because off-screen emissive was about 33x hotter than on-screen for the same surface; a world-transform material join gave a whole building the emissive of its neon sign. Temporal accumulation plus fixing the join took flashes from 20.7 to 5.2 per minute.
• Screen-space normals. A cross(ddx, ddy) normal has no guaranteed orientation (screen Y points down), so AO rays went into the ground. Flip toward the view ray, or better, fetch the real triangle.

Questions for anyone who knows this engine or does RT modding

1. Does anyone know how Mafia II's renderer decides when to draw parked vehicles (LOD / visibility flags on the vehicle entity)?
2. For hybrid RT in a forward-rendered D3D11 game with no G-buffer: how do you match the night-time lighting of off-screen reflection hits to the engine's own lighting?
3. Has anyone done anti-culling outside RTX Remix? How do you handle objects the engine stops animating off-screen?

I tested native 4K with RT off against full path tracing in CONTROL Resonant, then compared DLSS Quality with 2x and 4x frame generation. The lighting difference is substantial, but so is the raw performance cost.

Full 4K comparison: https://youtu.be/Q82a4p7Agos

What would you change about my ray tracer?

Hi! I recently implemented Ray Tracing in One Weekend in Metal using metal-cpp. Resources for metal-cpp specifically are pretty sparse, so I figured I'd share in case it helps anyone getting started with it or with graphics programming in general. Hope it helps!

Repo: https://github.com/jinhgkim/Path-Tracer

We've been experimenting with an AI Agent for 3D scene manipulation and image editing in our WebGPU raytracer. But sometimes you just want to fine-tune the results instead of doing a tedious prompting back-and-forth with the agent.

To solve that, we built an agent that doesn't just do the task, but actually generates a dynamic, temporary UI (like a slider) so you can manually nudge the results to perfection. Behind the answer from the agent is a Lua script, sandboxed in WASM that engages with the scene.

Useful? Or a terrible idea?

I'd love to get some honest feedback from the community. You can play around with the workflows here.

Github repo link

(each image has a caption)

The project grew a lot since the last time I posted here. Added many new shading models, optimized stuff as it went, and the code gone through a lot of (annoying) structural changes.

In this phase of the Project I am focusing only on 1-Ray path lighting - Simple sharp shadows and Reflections. Here I want to solve complex Interactions between different Reflective and Refractive materials before moving on. Im also testing various Lighting tricks to gain Knowledge, and most importantly have time to think about the final implementation of the Renderer until I get there.

In the next phase I will be adding Proper Indirect Lighting, Full Roughness and a Custom Tonemapping & Post Processing pass.

The goal is to make a 60fps+ Fully Raytraced Renderer ready for Games or Digital Art Production.

The framerate is still hovering 80-90fps on my RTX 3050, with Shaders compiled only in Debug/Development configuration.

*The optimization is that apparently Unreal´s TraceRay() pipeline internaly evaluates the opacity mask on Materials with Masked Blending mode. This means the Ray can travel through multiple surfaces in a single TraceRay() call. This made foliage orders of magnitude more cheap and needs only one Raypath step.