Suppressing unwanted content without negative conditioning, cross-scene consistency in 3DGS, and data-driven artwork placement on generated walls

[Difacto]

Practical problems from a production project, would appreciate pointers to literature or working practice.

SETUP 11-scene walkable environment, self-hosted WebGL (three.js + Spark). Pipeline: text prompt → Marble (World Labs) → equirect pano + .spz + collision mesh. Period-accurate European Renaissance, ~1500. The scenes host a rotating art exhibition.

INTERACTION MODEL (relevant to Q4 and Q6) Visitor walks in under ambient light. On approaching and stopping in front of a work, that single work brightens while the rest of the room is unchanged; a bench beside it becomes an offer to sit; sitting locks the camera to a fixed viewing pose; the work then opens full-screen with title, year, technique and real dimensions, pageable left/right. The previous version of this project was a hand-authored three.js scene with real lights, where this was trivial (pooled SpotLights + emissiveIntensity). Migrating to generated splats broke precisely this, since a splat carries baked illumination and exposes no lights.

PRIOR ATTEMPTS, so nobody suggests these again: - Blockade Labs Skybox: ~45 generations rejected. Photoreal quality and pole distortion in 2:1 equirect. Also failed as a structure- preserving upscaler (Remix at Influence 85 and 100 both replaced the scene rather than upscaling it). - ChatGPT image agent: correct period, capped at 1774 px. - Midjourney: no native equirect, 12 images dropped. - DiT360: promising anti-seam/anti-pole work, but 2048x1024 only. - Nano Banana Pro / GPT Image 2 / FLUX.2 Pro: no equirect support.

QUESTIONS

1. NEGATIVE-FREE SUPPRESSION Marble exposes no negative prompt and does not respond to negation in the positive prompt. It consistently introduces anachronisms: electrical outlets, gilt-framed paintings on walls specified as bare, religious figures, Baroque furniture. Is there established technique for suppression under these conditions - attention manipulation, prompt-space steering, anything applicable to a closed API where I only control the text?

2. CROSS-SCENE CONSISTENCY Fixed seed plus shared prompt core is insufficient; scenes read as different buildings. Is there work on conditioning multiple generations on a shared latent or reference set? Is single-large- scene generation followed by segmentation the more sound approach?

3. UNBOUNDED SCENE QUALITY Indoor generations are strong, large-scale outdoor degrades in geometry and horizon coherence. Known limitation of image- conditioned 3DGS, and are there models specifically stronger on unbounded scenes? 4 of my 11 scenes are exterior.

4. RELIGHTING INSERTED GEOMETRY IN A BAKED SPLAT Flat textured quads (paintings) inserted into a splat read as decals: the splat carries baked lighting, the quad has none. Current empirical fix - measure the wall region’s mean luminance, HSV saturation and R/B ratio from the pano, pre-grade the artwork toward those targets, add slight blur to match splat frequency. Measured: wall 94 / 67 / 1.32, artwork 142 / 70 / 0.77, graded to 104 / 84. Visually much better but ad hoc. Is there a principled method - estimating an environment map or SH irradiance from the splat and relighting properly?

5. DATA-DRIVEN PLACEMENT ON GENERATED GEOMETRY The exhibition rotates monthly: up to 10 works per gallery across 11 scenes, ~130 rehangings a year, performed by a non-engineer. Placement therefore cannot be hardcoded.

   Complications: works span 30x30 cm to 200x160 cm (35x in area), mixed portrait/landscape/square. The wall geometry is generated, not authored - I raycast the collision mesh to find it. One wall measured as a plane at 66.7 degrees to the world axis with a protruding sconce mid-span; the room yields ~14 m of usable wall across 7 segments of 1.7-2.4 m, broken by windows and a fireplace.

   Current approach: extract free wall segments by raycasting the collider at three heights (1.10 / 1.50 / 1.90 m) and marking a segment usable only where all three hits lie on the same plane within 10 cm - this reliably separates flat wall from windows, recesses and protruding fixtures. Then place fixed square “slots” (1.0 / 0.7 / 0.45 m) in those segments once per scene. Monthly rotation becomes a list of work IDs mapped to slots; each work is aspect-fitted into its slot, so true physical scale is sacrificed and real dimensions are surfaced in the detail overlay instead.

   Two questions: is the three-height coplanarity test a reasonable way to segment usable wall from a noisy generated collider, and is sacrificing true scale the right trade for making rotation a data-only operation? Prior art on automated gallery hanging would be very welcome.

6. SELECTIVE ILLUMINATION INSIDE BAKED SPLAT GEOMETRY Following from the interaction model: I need one object to brighten on approach while its surroundings stay put. In authored geometry this is a SpotLight. In a splat there is nothing to light. Candidates I’ve considered: (a) raising emissive on the inserted quad alone, which crosses into looking like a backlit display since the wall does not respond; (b) additive cone geometry to fake spill onto the wall; © runtime per-splat colour modulation in a neighbourhood around the work - is that tractable at ~500k splats in a browser, and are there implementations; (d) inverting the problem by dimming the global exposure so the unmodified work becomes relatively brighter. Is there established practice for spotlighting within baked radiance-field geometry? Pointers to relighting work on 3DGS that is fast enough for interactive use would be ideal.

Happy to share measurements, prompts, or a demo. I’m a designer working with AI coding assistants - papers welcome, but I’ll need the practical version alongside.

[John6666]

Hmm… for now, based on what I could find, this seems to be roughly how things stand:

---

I think all six problems are real, but they sit at slightly different layers, and that changes what is worth trying.

The short version would be:

Question	What seems most useful in practice
1. Suppression without negative prompting	With a closed API, most attention/CFG-based papers are not directly actionable. Marble’s own panorama edit stage is probably the first practical route; for API-only use, disable_recaption is at least a cheap controlled A/B.
2. Cross-scene consistency	The literature seems to favor shared spatial structure/memory, not merely a shared random seed. Floorplans, coarse geometry, references, or continued generation look more promising than trying to make independent scenes agree after the fact.
3. Outdoor/unbounded degradation	Unbounded generation really is treated as a separate hard problem in recent work, but I would first locate which pipeline stage is degrading before blaming 3DGS in general.
4. Inserted artwork looks like a decal	First rule out a Three.js color-management/material mismatch. After that, I would keep your current lightweight grading as a baseline and try a local environment probe before attempting to recover physical illumination from SPZ SH coefficients.
5. Data-driven hanging	Your three-height test looks like a reasonable task-specific wall detector if it is already reliable. I would keep it, but make the output a stable occupancy/placement layer so monthly rehangs never need to understand the raw generated geometry again. True scale and data-only rotation are not necessarily mutually exclusive.
6. Selective spotlighting in splats	Your option (c) is very close to something Spark already implements: SplatEdit supports spatial RGBA edits, including an INFINITE_CONE explicitly documented as a spotlight-like primitive. This is probably the first thing I would test.

If I were trying to minimize engineering cost, my order would be:

1. verify the Three.js color pipeline for the paintings;
2. inspect whether the actual SPZ has any SH beyond degree 0;
3. prototype one Spark SplatEdit spotlight in one representative room;
4. separate Q3 into pano → Marble world/mesh → exported SPZ → Spark;
5. leave the working wall-segmentation heuristic alone until it produces an actual failure case;
6. only then look at heavier inverse-rendering / relighting methods.

The common pattern I see is that it may help to separate three representations:

• the generative representation used to create the space;
• the visual representation used to render the splats;
• the operational representation used by the exhibition system.

The monthly hanging system does not really need to understand Marble’s raw geometry every month, and the painting renderer does not necessarily need to recover the physically correct latent lighting of the splat if a stable local appearance approximation is enough.

1. Negative-free suppression: closed API changes the useful solution space

There is a substantial literature on negative guidance, attention manipulation, CFG modifications, and related techniques, but most of it assumes access to the diffusion process itself. That makes it useful as background evidence that negation/content suppression is genuinely non-trivial, but not necessarily useful implementation advice for a Marble text-only API.

For a closed model, I found two more relevant directions.

A. Fix the generated panorama before committing to the 3D world

World Labs’ current Create & edit workflow explicitly separates:

1. panorama generation;
2. optional panorama editing;
3. draft 3D generation;
4. final world generation.

At the panorama-edit stage, the docs describe targeted local edits and specifically list modifying objects, adding/removing details, and fixing issues in the initial generation.

For persistent things like:

• outlets,
• an unwanted framed painting,
• a religious figure,
• a wrong piece of furniture,

that looks more reliable to me than trying to encode increasingly complicated negations into the original positive prompt.

It also has a nice engineering property: the correction occurs before the 2D panorama is lifted into the final world, rather than trying to repair the splats later.

B. If the workflow must remain API-only, test recaptioning separately

The current World API exposes both a generation seed and a disable_recaption field for text prompts in the world generation API.

I would not assume recaptioning is causing the anachronisms — I found no evidence for that — but this gives a cheap controlled test:

same model
same seed
same text_prompt
only change disable_recaption

If the unwanted objects are unchanged, that hypothesis can be discarded quickly.

C. Black-box prompt refinement exists, but I would rank it below editing

There is also work such as Test-time Prompt Refinement for Text-to-Image Models, where a black-box generator produces an image, a multimodal model checks the result against the prompt, and the prompt is rewritten for another round.

Conceptually:

prompt
  ↓
generation
  ↓
detect unwanted/missing content
  ↓
rewrite prompt
  ↓
regenerate

This is much closer to your “I can only control text” constraint than attention manipulation is.

But I would still treat it as a third-line option here:

• the paper is about T2I, not Marble;
• every iteration costs another generation;
• you need a reliable evaluator for period errors;
• Marble already exposes an edit stage designed for local corrections.

So my default route would be targeted panorama editing where available, API A/B diagnostics if not, and only then automated black-box prompt optimization if the number of worlds makes that worthwhile.

2. Cross-scene consistency: seed consistency vs spatial consistency

This is the area where the literature seems most consistent.

A fixed random seed can make sampling reproducible, but it does not by itself provide a persistent representation of:

• the building’s proportions;
• room adjacency;
• wall/floor material identity;
• window grammar;
• architectural details;
• where previously generated structure physically exists.

Recent whole-scene / multi-room systems tend to add exactly that missing state.

A particularly close example is PanoWorld, which targets consistent whole-house panorama synthesis. Its design uses:

• a floorplan-derived 3D shell as global structural guidance;
• a dynamic 3DGS cache as persistent spatial/visual memory.

That is much closer to “the next room knows what building it belongs to” than sharing a seed.

I would use PanoWorld mostly as evidence for the design pattern, not as a drop-in implementation: the repository currently exposes the PanoWorld-LRM inference path, while several components of the full generation pipeline are still listed as forthcoming.

Marble already exposes some controls in the same general direction

Chisel lets you block out coarse 3D geometry first and use it as the foundation for the generated world.

Expand continues outward from an existing world and is explicitly intended to preserve visual style, architecture, scale, and continuity at the connection.

That seems much closer to the cross-scene problem than generating eleven independent rooms and hoping their latent identities coincide.

There is one current constraint worth noting: the docs say a world generated with Marble 1.1 Plus cannot currently be expanded.

Studio Compose is another option, but it is useful to distinguish composition from generation consistency. The docs describe a multi-room house as independently generated room worlds that you manually position, rotate, scale, align, and connect; they even include “match lighting” as a connection step. So Compose solves assembly, but does not magically make independently generated rooms share architectural identity.

So I would frame the options like this

If independent worlds are a hard product requirement:

Keep the worlds separate, but increase shared evidence: common reference imagery, common coarse geometry, repeated spatial constraints, or some canonical authored layout.

If one connected generation is acceptable:

Prefer generation that grows from an already established structure/world over independent regeneration.

If neither is possible:

Treat consistency as an explicit post-generation acceptance criterion, rather than expecting a seed to enforce it.

I would therefore be cautious about “single huge scene then segment” as the answer. It is one way to force a shared spatial state, but the broader principle seems to be persistent structure/memory, not necessarily one monolithic splat.

3. Outdoor / unbounded scenes: real research problem, but isolate the failing stage first

There is good evidence that unbounded 3D generation is not just “indoor generation, but larger”.

For example, VideoRFSplat explicitly targets unbounded real-world scenes and jointly models multi-view imagery and camera pose.

GaussianCity similarly treats scaling 3DGS from finite scenes to unbounded city-scale environments as non-trivial, requiring a dedicated compact representation rather than simply allowing the point set to grow without bound.

Those papers do not establish the cause of your four bad outdoor scenes — their training setups and objectives are different — but they do support the narrower statement that large/unbounded generation has its own geometry, memory, pose-consistency, and representation problems.

For your pipeline, I think the higher-value test is to ask where the failure first appears.

The World API exposes the panorama, multiple mesh assets, splat assets, and hosted Marble world separately, so the stages can be compared.

Is the horizon/geometry already wrong in the panorama?
    ↓ yes
generation / panorama-side problem

Panorama looks correct, but Marble 3D world is wrong?
    ↓
3D lifting / world reconstruction side

Marble world looks correct, but exported SPZ/mesh is wrong?
    ↓
export / representation side

Exported asset looks correct elsewhere, but Spark is wrong?
    ↓
runtime / renderer side

That distinction matters because “3DGS cannot do outdoor scenes well” would be much too broad if the pano itself is already inconsistent, and equally misleading if the asset is good until it reaches the browser renderer.

World Labs currently describes Marble 1.1 Plus as its model for the largest worlds, automatically expanding 3D coverage where possible. So if your exterior scenes were generated with another model, a small 1.1 vs 1.1 Plus comparison is a reasonable branch.

I still would not assume Plus fixes horizon coherence; the documentation only establishes that it is intended for larger coverage.

One Spark-specific branch is also worth keeping in reserve: current SparkRenderer documentation includes pagedExtSplats, described as useful for avoiding quantization artifacts when splat scenes have very large internal position coordinates. That is a renderer-side precision tool, not a general fix for bad generated geometry, so I would only investigate it if the degradation appears specifically after loading into Spark and looks coordinate/precision-related.

4. Relighting the paintings: I think there are several cheaper steps before inverse rendering

Your current fix does not look unreasonable to me.

You effectively have a baked visual field, then insert an object that was never present when the field’s appearance was generated. The object does not inherit the wall’s baked exposure, color cast, local softness, or frequency characteristics, so some kind of appearance harmonization is expected.

There are papers treating the heavier version of this exact problem. GauUpdate explicitly observes that inserting new Gaussian objects into an existing Gaussian field gives inconsistent appearance when the source and target lighting differ, and solves it through inverse rendering of materials/environment illumination.

D3DR instead uses diffusion priors to harmonize inserted 3DGS objects, including lighting and shadows.

Those are useful evidence that the mismatch is a real research problem, but they are much heavier than your flat artwork use case.

I would try the following ladder first.

Step 0: verify this is not partly a Three.js color-pipeline mismatch

Before estimating illumination, I would check the ordinary renderer plumbing.

Three.js’ Color Management guide distinguishes:

• sRGB input color textures;
• Linear-sRGB working/rendering space;
• output color conversion / tone mapping.

For normal PNG/JPEG artwork textures, the color texture should generally be tagged appropriately as sRGB color data. If the painting quad, splat renderer, and post-processing path are going through different color transforms, a brightness/color mismatch can look deceptively like a lighting mismatch.

I would check at least:

• texture.colorSpace;
• renderer.outputColorSpace;
• tone-mapping settings/exposure;
• whether post-processing performs the final output transform;
• whether the artwork uses MeshBasicMaterial, MeshStandardMaterial, a custom shader, etc.

In particular, if the painting is effectively unlit (MeshBasicMaterial or equivalent), a physical Three.js light will never make it inherit the same illumination behavior as the room.

If the color pipeline is correct and the mismatch remains, then I would call it the baked-vs-inserted appearance problem.

Step 1: keep your current empirical grading as the baseline

Mean luminance, saturation, channel ratio, and a little blur are all very cheap and directly optimize what you actually care about: visual integration.

I would keep that baseline even if you later add something more principled, because it gives you an easy A/B:

does the more complicated method visibly outperform the simple wall-statistics transform?

There is also one museum-specific constraint I would keep separate from generic “object harmonization”: you probably do not want the harmonizer to materially change the painting itself.

For an ordinary inserted chair, changing the object’s chroma or tonal range to fit the room may be fine. For an artwork, aggressive harmonization can defeat the point of showing the artwork accurately.

So I would separate:

• faithful painting image;
• frame/glass/carrier geometry;
• local wall response.

That gives you more freedom to harmonize the frame and surroundings while preserving the artwork pixels.

Step 2: Spark can render a local environment probe directly

This was the most interesting practical thing I found for Q4.

The current SparkRenderer documentation exposes:

• renderCubeMap(...);
• readCubeTargets();
• renderEnvMap(...).

renderEnvMap() renders the splat scene from a supplied world position, builds the six cube faces, prefilters them using Three.js PMREMGenerator, and returns a texture that can be assigned directly to MeshStandardMaterial.envMap.

So instead of:

decode SPZ SH → infer the original environment lighting

you can potentially do:

render the actual baked scene appearance around the painting position → use that as a local image-based-lighting probe for the inserted frame/carrier.

That is not a reconstruction of the true physical light field. It is a local appearance probe derived from what the viewer actually sees, which may be exactly the useful quantity for this application.

readCubeTargets() also means you can retrieve the six rendered faces as RGBA buffers. If you wanted to extend your current luminance / R:B heuristic, you could estimate low-frequency local exposure/tint from a 360° neighborhood rather than from only the wall patch.

I would personally try this before attempting to interpret SPZ SH as irradiance.

Why I would be cautious about the SH route

The current Niantic SPZ implementation allows sh_degree from 0 to 4.

Degree 0 means there are no additional SH coefficients at all, and the source describes the SH field as coefficients for view-dependent colors.

So there are two separate questions:

1. does the particular Marble SPZ actually contain non-zero-degree SH?
2. if so, what physical quantity can legitimately be inferred from it?

The standard SPZ structure gives you Gaussian geometry, alpha, base color and optional view-dependent color coefficients. It does not give you a clean, separately identified BRDF + surface normal + incident illumination decomposition.

Relightable-3DGS papers generally have to estimate or learn those additional factors precisely because they are not already handed to you as a standard splat asset.

So I would inspect the file’s sh_degree, but I would not design the production system around “SH = environment light”.

Step 3: if local probes + grading are not enough, use a proxy-mesh relighting approach

There is a very relevant production precedent in PlayCanvas’ Gaussian Splat Relighting.

Their method is roughly:

1. use a simplified mesh approximating the splat;
2. light that mesh with ordinary dynamic lights;
3. render the mesh lighting to an offscreen texture from the active camera;
4. modulate the splat fragments using that lighting texture.

That is a clever bridge between ordinary real-time lighting and a baked splat.

You already have a Marble collision mesh, so it might be worth checking whether it is geometrically close enough to the visible wall surface to serve as that sort of proxy.

I would treat that as a test, not an assumption: PlayCanvas notes that proxy/splat alignment is an important quality factor.

So for Q4 my preferred escalation path would be:

Three.js color sanity
    ↓
your current appearance grading
    ↓
local Spark env-map probe
    ↓
proxy-mesh lighting transfer
    ↓
inverse rendering / diffusion harmonization

That seems much cheaper than jumping directly from wall RGB statistics to full inverse rendering.

5. Artwork placement: I would preserve the current heuristic, but formalize its output

For the first question — whether the three-height coplanarity test is reasonable — I think yes, as a task-specific heuristic, provided the observation that it reliably rejects windows/recesses/fixtures holds across your rooms.

I could not find anything saying that exactly:

• 1.10 m;
• 1.50 m;
• 1.90 m;
• ±10 cm

is a standard gallery-wall algorithm.

So I would not present those particular constants as generally established.

But the broader idea is well connected to indoor-geometry processing: using evidence across multiple horizontal slices/heights is a normal way to make wall detection less sensitive to clutter, occlusion, and local protrusions.

Given that your purpose is not “reconstruct the mathematically perfect wall plane”, but “find conservative regions where a painting will not intersect architectural clutter”, your heuristic is arguably solving the right problem.

I would not replace it with RANSAC unless you have an actual failure

If the current procedure works, replacing it with semantic segmentation, a full mesh classifier, or global plane fitting may increase complexity without increasing useful information.

A reasonable escalation would be only when a bad segment is observed:

three-height agreement
    ↓ failure case?
local plane residual
    ↓ still ambiguous?
surface-normal coherence
    ↓
test the whole artwork rectangle for clearance

That keeps the cheap detector in the common case.

The more important architectural step is what happens after detection

I think your “compute once per scene, rotate data monthly” idea is the right boundary.

A useful comparison is OpenVGAL, an open-source virtual gallery project.

Its authored gallery templates contain Occupancy_* planes: simple strips saying, effectively, “art can be placed here”. The gallery generator extracts those strips and then performs width-aware artwork packing from data.

Your case is harder because the room was generated, so a human did not author the occupancy planes.

But the architecture maps almost perfectly:

your generated collider
        ↓
three-height / plane tests
        ↓
stable usable-wall strips
        ↓
monthly artwork metadata
        ↓
packing / slots

In other words, your current raycast step can be viewed as an automatic front-end that generates the occupancy abstraction an authored gallery would normally supply manually.

Once those strips exist, I would try hard to keep the monthly system completely independent from the raw collider.

True physical scale does not have to conflict with data-only rotation

This is where I would slightly modify the current design choice.

Your fixed square slots are a very defensible production compromise, especially if the full-screen overlay gives the true dimensions.

But data-only operation does not require sacrificing scale.

OpenVGAL’s current layout stores artwork width/height in real-world centimetres and packs them into available occupancy strips by width.

There is also academic work on automated virtual-gallery layout, for example Space-adaptive Artwork Placement Based on Content Similarities for Curating Thematic Spaces in a Virtual Museum, which treats artwork placement as an optimization over spatial constraints rather than hard-coded manual transforms.

So you could support three modes without changing the scene authoring boundary:

Fixed-slot mode

• what you have now;
• simplest;
• very predictable.

Physical-scale mode

• store real dimensions in metadata;
• place at true scale if the strip has capacity;
• overflow to another strip when necessary.

Hybrid mode

• preserve true scale for works where scale is curatorially important;
• slot-fit the rest.

That turns “true scale or data-driven operation” into a policy choice rather than a technical limitation.

For a monthly rotation system I would probably still retain fixed slots as the default, because predictability is valuable. But the stable occupancy layer gives you room to add scale-aware packing later without reauthoring 11 scenes.

6. Selective illumination: Spark already has a lightweight version of option (c)

This one seems the most directly actionable.

Spark’s current Splat Editing documentation says that SplatEdit applies RGBA/XYZ fields to splats as part of the normal SplatMesh pipeline.

For color it currently exposes:

• MULTIPLY;
• SET_RGB;
• ADD_RGBA.

And the available spatial SDFs include:

• sphere;
• box;
• ellipsoid;
• cylinder;
• capsule;
• INFINITE_CONE.

The documentation explicitly describes spheres as useful for point-light-like effects and infinite cones as useful for spotlight-like effects.

So your option (c) is not only tractable in principle; Spark already exposes the basic building blocks for it.

There is also a first-party Dynamic Lighting example that loads an SPZ scene, creates SplatEdit lighting layers, applies spherical ADD_RGBA regions, and changes the colors over time in the animation loop.

For your use case, conceptually I would start with something like:

// sketch only — tune position/orientation/falloff for each wall
const spill = new SplatEdit({
  rgbaBlendMode: SplatEditRgbaBlendMode.ADD_RGBA,
  softEdge: 0.4,
});

const cone = new SplatEditSdf({
  type: SplatEditSdfType.INFINITE_CONE,
  color: warmLightColor,
  opacity: 0,       // avoid changing splat opacity
  radius: coneWidth,
});

spill.add(cone);

// Prefer attaching/scoping the edit to the room SplatMesh
// rather than unintentionally editing every splat in the scene.
roomSplat.add(spill);

I would keep the RGB contribution fairly restrained. Spark’s documentation explicitly warns that ADD_RGBA can become hyper-saturated, and adding non-zero alpha can make previously low-opacity splats more opaque.

That matters here because the goal is not to make the wall glow; it is to create just enough local response that the painting no longer reads as a self-illuminated display.

Performance: 500k is not enough information by itself

Spark’s performance guide gives broad platform budgets in the millions of splats, but also makes an interesting warning: even about 500k splats concentrated in a small screen area can become a GPU rendering/blending bottleneck.

So I would not estimate this from splat count alone.

The lowest-cost benchmark is probably just:

same room
same camera poses
same pixel ratio

A: SplatEdit disabled
B: one soft cone enabled

and compare frame/GPU time at:

• normal walking distance;
• close to the painting;
• an oblique angle where many splats overlap in screen space.

That will tell you more than a theoretical “500k should be fine”.

One small current-version trap

The Spark repository’s Dynamic Lighting example currently contains an ambient layer using SplatEditRgbaBlendMode.DARKEN, while the current SplatEditRgbaBlendMode source and documentation list only:

• MULTIPLY;
• SET_RGB;
• ADD_RGBA.

So I would use the current documented enum as the API contract rather than blindly copying every line of that example. The ADD_RGBA lighting part itself matches the current API.

If the fake spotlight is visually sufficient, I would stop there

This seems important.

Your interaction requirement is:

one artwork brightens as the visitor approaches, with a believable hint that its surrounding wall responds.

That does not necessarily require recovering a physically correct time-varying radiance field.

A subtle SplatEdit cone plus controlled artwork brightening may satisfy the perceptual requirement at a tiny fraction of the complexity of relightable 3DGS.

If it does not, then I would move up one level to the proxy-mesh approach mentioned under Q4.

Only if the exhibition genuinely needs:

• correct cast shadows;
• consistent normals/material response;
• large lighting changes;
• arbitrary moving light sources;

would I start looking seriously at methods such as GauUpdate, GS inverse rendering, or diffusion-based 3DGS harmonization.

Putting it together

I think the useful design boundary is roughly this:

                GENERATION TIME
                      │
      prompt / references / Chisel / Expand
                      │
                      ▼
              Marble world + pano
                      │
          ┌───────────┴───────────┐
          ▼                       ▼
    visible SPZ              collision mesh
          │                       │
          │                       ▼
          │              usable wall strips
          │                       │
          │                 stable slot/packing
          │                       │
          └──────────┐     monthly artwork data
                     │             │
                     ▼             ▼
                  RUNTIME EXHIBITION
                     │
        ┌────────────┼─────────────┐
        ▼            ▼             ▼
 artwork texture   local probe   splat spill
 / frame material  / grading     via SplatEdit

The part I like about this separation is that none of the work you have already done needs to be thrown away.

• The three-height test can remain the cheap front-end for deriving wall occupancy.
• The current artwork grading remains a useful baseline.
• Fixed slots can remain the default while leaving room for true-scale packing later.
• Your interaction model can be restored approximately with existing Spark machinery.
• Cross-room generation consistency can be attacked upstream with more shared spatial evidence rather than by complicating the exhibition runtime.

If I had to pick only three experiments before doing anything more ambitious, I would choose:

1. Q4: make sure the painting texture/material is in the same Three.js color-management/tone-mapping pipeline as the rest of the render;
2. Q6: try one soft INFINITE_CONE SplatEdit on one representative 500k room and measure it;
3. Q2: compare one pair of independent rooms against one pair produced with genuinely shared structural context (Chisel/reference/continuation), rather than another fixed-seed prompt variant.

Those three should answer fairly quickly whether the remaining problems require research-grade methods or whether the production-friendly approximations are already enough.

[Difacto]

Thank you so much for your detailed reply, dear John6666. I’ve been so desperate for four weeks now that I think I need to approach this completely differently… I’m a graphic designer, but not a developer, but I love this world… Perhaps you have some ideas on how to start with something like this… without constantly having to rename projects a hundred times because something doesn’t fit. I’m incredibly grateful for any tips.

[John6666]

Hmm… maybe it would be better to first figure out a way to experiment without losing your bearings?:

---

Your follow-up changes how I would answer this.

After four weeks of wrestling with this, I would pause the six original questions for a moment and make the experiments themselves cheap to try, easy to compare, and easy to undo.

That seems more useful than adding another layer of architecture right now.

I also would not assume “renaming projects a hundred times” points to one particular underlying problem.

If you literally mean folders/projects turning into things like:

museum-final
museum-final2
museum-lighting-test
museum-lighting-test-fixed
museum-real-final

then yes, version control is designed to replace that kind of history-by-filename.

But if you mean something broader — an AI-assisted change keeps turning into a different project, or after enough edits you no longer know which world / .spz / collider / code state actually belongs together — Git alone will not solve that.

Because you are coming to this from graphic design and using AI coding assistants, I would not turn the answer into “first, become a developer.”

I would give the project a very small memory outside the chat.

The whole idea would be:

Keep one place called HOME, change one main idea at a time, and always know how to get back.

A checkpoint is a little like taking a photo before rearranging a room. You are still free to move everything around; the point is that “before” never becomes ambiguous.

If I were starting tomorrow morning

I would put five small guardrails around the project:

	The small version
1. HOME	Pick one working room you understand and record the code + Marble/SPZ/collider/transform state that belongs to it.
2. Visible history	Make one untouched backup; if folder/project copies are multiplying, let Git history replace final-final2 naming.
3. Project note	Keep one tiny PROJECT_STATE.md: current question, what stays fixed, how you will check it, what happened, and what you meant to do next.
4. One-room laboratory	Test a new idea in one representative room first. The other ten scenes are the later “does this generalize?” test.
5. AI workbench	Let the chat explore, but keep HOME, decisions, tried ideas, and next action in the project rather than only in the conversation.

That is the system in miniature.

If those five lines already make sense, I would start there and ignore the rest until something specific hurts.

The concrete version — including the tiny files/checklists I mean — is below.

The concrete HOME / Git / PROJECT_STATE / AI setup

1. Pick one room and call it HOME

Not the most beautiful room.

Not the final architecture.

Just the working room you understand best.

HOME means:

“If everything gets confusing, I can return here and I know what this state is supposed to do.”

For your project I would write down, at minimum:

HOME ROOM:
____________________

CODE CHECKPOINT:
____________________

MARBLE WORLD / VERSION:
____________________

SPZ:
____________________

COLLIDER:
____________________

PLACEMENT / TRANSFORM DATA:
____________________

THREE.JS / SPARK VERSION:
____________________

That is enough to establish a coordinate on the map.

And I would be slightly conservative about moving HOME.

A candidate becomes the new HOME only when:

the intended thing improved
+
the little regression checks still pass
+
you have recorded which asset/dependency state belongs to it

If it merely looks promising, keep it as a candidate for now.

That avoids a subtle version of getting lost where HOME itself moves every time an experiment looks interesting.

A project name is not quite enough here, because your visible room is really a bundle of independently changeable things.

2. If project copies are part of the problem, give the code a visible history

Before changing the workflow, I would make one untouched backup copy of the current working folder and then stop editing that copy. That is simply the emergency return point while you set up a better history.

After that, I would probably use GitHub Desktop if command-line Git is not something you want to learn right now.

It lets you make a visible checkpoint (“commit”) and later inspect the history of exactly what changed.

The Changes view is useful with AI coding in another way too: before accepting a checkpoint, scan the changed-file list. If you asked for a spotlight experiment and suddenly see the collider loader, camera system, package configuration, and placement code changing too, that is a good moment to ask why.

The repository can stay local on your computer; publishing it is optional, and GitHub Desktop can publish it privately later if you want (walkthrough).

Before the first commit, make sure API keys, tokens, passwords, .env files, etc. are not entering history; GitHub recommends excluding such files and not hard-coding secrets (guidance).

For now I would not start with:

• complicated branching strategies;
• rebasing;
• pull requests;
• CI pipelines;
• issue-tracker rituals.

You can learn those if you actually need them.

The useful minimal rule is much smaller:

Make a checkpoint whenever you reach a state you might want to return to.

Names such as:

gallery-03 baseline works
painting placement works
before spotlight experiment
spotlight prototype works
before room-switch cleanup

tell you much more than v17 or final-final2.

One caveat: large .spz / .glb histories do not necessarily belong directly in ordinary Git. Initially I would simply record which exact asset files belong to the code checkpoint. If large binary history later becomes a real problem, Git LFS is one possible next step.

3. Keep one tiny PROJECT_STATE.md

This is the part I would consider more important than Git.

Something like:

HOME
gallery-03 / checkpoint: __________

CURRENT QUESTION
What am I trying to learn?
___________________________________

ONE MAIN CHANGE
___________________________________

KEEP FIXED
What should NOT be redesigned in this experiment?
___________________________________

HOW I WILL CHECK
___________________________________

RESULT / LEARNED
___________________________________

NEXT INTENDED ACTION
___________________________________

IF THIS FAILS, RETURN TO
___________________________________

PARKED IDEAS
___________________________________

Before you stop for the day, leave the next intended action there.

It is the project equivalent of leaving yourself a note on the desk saying, “I stopped here; this is what I meant to do next.”

There is even research on interrupted programming work showing that developers use contextual cues and notes to reconstruct where they were when they resume: Evaluating Cues for Resuming Interrupted Programming Tasks.

And I would keep rejected ideas very briefly too:

TRIED:
RESULT:
WHY I STOPPED:
RETRY ONLY IF:

That prevents a fresh AI conversation from rediscovering last week’s dead end and presenting it as a new idea.

I would also save maybe three fixed screenshots of HOME:

G1: entrance / whole-room view
G2: one painting at interaction distance
G3: seated/fixed viewing position

When an experiment changes something visual, capture the same three views again.

You do not need automation: HOME/ and candidate/ screenshot folders already give you a visual before/after. If it later becomes worthwhile, Playwright can maintain visual baselines; its docs also warn that rendering varies by environment, so treat them as controlled comparisons, not absolute truth.

4. Make one room the laboratory

You have 11 scenes. I would not use all 11 to answer the first version of a question.

Think:

one room = laboratory
the other ten = “does this generalize?” later

You would not test a new paint mixture on every wall in a museum at once.

For example:

• Q4 relighting → one painting, one wall, one fixed camera;
• Q5 placement → one wall with windows/recesses/fixtures;
• Q6 spotlight → one painting, one Spark edit;
• Q2 consistency → two rooms, not eleven;
• Q3 outdoor degradation → one exterior;
• Q1 unwanted generation → one prompt / one candidate world.

This is probably the biggest change I would make to the way the six original questions are approached.

They do not have to become one giant architecture problem.

Each can be a small experiment that starts from HOME and either earns its way into the project or gets discarded.

5. Treat the AI chat as a workbench, not the archive

A workbench is where things get spread out, tested, taken apart and sometimes abandoned.

That is useful.

But I would not let the chat be the only place that knows:

• what HOME is;
• which asset bundle belongs to it;
• what has already been tried;
• what the AI is allowed to change;
• what the next intended action is.

Those belong somewhere durable in the project.

Before a substantial AI-generated edit I would give it a small contract like this:

GOAL
Prototype one local spotlight in gallery-03.

ALLOWED TO CHANGE
- spotlight prototype code
- one scene hook if necessary

KEEP FIXED
- Marble world / SPZ / collider
- coordinate conversion
- artwork placement
- camera interaction
- data schema

BEFORE EDITING
Tell me which files you intend to change and why.

AFTER EDITING
Tell me:
1. which files actually changed;
2. what behavior changed;
3. how I can compare this with HOME;
4. whether you discovered a reason a protected boundary must change.

Do not automatically continue into a second redesign.

If it discovers that the experiment really does require changing a protected part, that becomes a new explicit decision rather than an invisible side effect.

If those five habits already make the project feel more legible, I would stop there for a while. The rest is only for when a specific kind of confusion still survives.

---

Three questions that tell me it is time to stop branching

At any point, ask:

1. Which exact state is HOME?
2. What single question is the current experiment answering?
3. What observation would make me keep this change rather than return HOME?

If one answer becomes “I’m not sure,” stop adding changes for a moment.

Not forever. Just long enough to recover the map.

This is not only a beginner phenomenon: studies of experienced developers describe disorientation when navigation context disappears, views are revisited repeatedly, or side tasks displace the original task (field study, follow-up).

So I would treat orientation as something the workflow should preserve.

The nice thing is that you do not need to know why you became lost before you recover.

Google’s incident-response guidance makes a similar separation: restoring/mitigating toward a known-good state and understanding the complete root cause can be different jobs (Incident Response).

---

Then I would turn the six original questions into six small experiments

Give each one its own small card rather than letting one AI conversation redesign the whole museum around all six.

The card can use the same five lines every time:

QUESTION:
KEEP FIXED:
CHANGE:
KEEP IT IF:
GO HOME IF:

That last line matters. A stop condition prevents “one more tweak” from quietly turning a small experiment back into a project rewrite.

Original question	Smallest experiment I would run
Q1 — unwanted generated content	One generation requirement + explicit forbidden-content checklist. Compare a small number of prompt/reference/edit strategies; accept/reject the result rather than endlessly modifying the whole pipeline.
Q2 — cross-scene consistency	Two representative rooms. Compare one stronger shared-state strategy — e.g. shared reference/geometry, Marble Expand, or explicit Compose alignment — against independent generation.
Q3 — exterior degradation	One bad exterior. Check where the problem first appears: pano → Marble world → exported SPZ/mesh → Spark.
Q4 — painting looks like a decal	One painting. First verify Three.js color handling; then compare your current grading against one local environment-map approach.
Q5 — automatic hanging	One difficult wall. Validate the existing three-height test against a small set of manually judged “usable / unusable” positions before making the algorithm cleverer.
Q6 — selective spotlight	One painting in HOME. Prototype one local splat edit and measure visual result + frame time before integrating it with the rest of the interaction system.

That changes the mental model from:

"I am rebuilding a complicated 11-scene system"

to:

"I am answering one small question,
then deciding whether its answer belongs in HOME."

And every experiment only needs one of three endings:

KEEP IT  -> it worked + the small checks still pass -> it can become HOME
GO HOME  -> it failed or answered the question negatively -> record that and return
PARK IT  -> interesting but unclear -> save the idea without building on top of it

PARK IT is especially useful with AI: a plausible idea can be remembered without becoming code today.

If you want the current Marble/Spark shortcut for each experiment

For Q2, Marble now gives you several different ways to share more state than a seed:

• Chisel lets you block walls/doorways or import GLB/FBX geometry, so the experiment can hold structure fixed while Marble supplies detail.
• Expand grows from an existing world and preserves earlier versions if the expansion is bad.
• Compose explicitly lets you join separate worlds by positioning, rotating and scaling them, then check floor levels, lighting and transitions.

Those answer different consistency hypotheses. I would test one, not “adopt Marble Studio.”

For Q4, there is now a pleasantly concrete Spark experiment: current SparkRenderer exposes renderEnvMap({ scene, worldCenter }). It updates/sorts the splats around a chosen point, renders six cube faces, prefilters them with Three.js PMREM, and returns a texture that can go directly on MeshStandardMaterial.envMap.

That is much cheaper to test than jumping directly into inverse rendering. I would call it an appearance-matching probe, not recovered physical illumination.

For Q6, current Spark SplatEdit explicitly describes MULTIPLY/ADD_RGBA as simple-lighting effects and INFINITE_CONE as a spotlight-like region. That makes:

one room
one painting
one cone
one intensity

an unusually clean first experiment.

I would benchmark it on the actual target machine, though. Spark’s own performance guide notes that even roughly 500k splats concentrated in a small screen area can bottleneck transparent blending despite larger overall desktop splat budgets.

For Q5, I would not throw away your existing wall detector just because it is heuristic. If the three-height coplanarity test is already separating walls from windows/recesses reliably, validate it first. Add normals/full-artwork clearance only if the mistakes you observe justify it.

The larger architectural win may be to turn accepted wall regions into a stable wall-local 2D layer:

generated collider
      ↓ preprocess once
wall_id + plane + usable intervals
      ↓
monthly artwork IDs / sizes

Then the monthly exhibition data no longer has to “understand” noisy generated geometry every time.

---

---

If you no longer know which version is HOME

Do not start by diagnosing the root cause.

Freeze first

Do not delete the current state. Save/checkpoint it.

Find the last state you can describe in plain language

For example:

“This room loads, I can walk, collision works, artwork placement works, and I know which SPZ/collider it uses.”

It does not need to be beautiful.

It needs to be reproducible and understandable.

If no such state exists, deliberately build the smallest one:

one room
one known SPZ
one matching collider
one camera path
one painting
no new experiment

Then rebuild one question

Not:

“fix the museum”

but:

“Does this specific spotlight approach work in one room?”

Only branch again after HOME exists.

A failed experiment is still useful if you know where it began, what it taught you, and where to return.

If folders/projects keep multiplying

If final, final2, new-final, etc. is literally the problem, this is the most direct route: let version history carry the history instead of filenames.

I would begin with only one habit:

checkpoint a working state before a risky AI change.

Good checkpoint names describe what became true:

gallery-03 baseline works
before Spark spotlight test
spotlight works, placement unchanged
before collider regeneration
room switching cleanup works

You do not need a branch for every thought.

You do not need to understand rebasing.

You do not need to publish the repository publicly.

The job of the history is simply to answer:

“What did this state contain, and can I get back to it?”

For large generated assets, keep the asset identity in the project map even if the binaries themselves are stored elsewhere.

If the AI keeps turning one fix into a redesign

This is where I would tighten the change boundary, not write a more elaborate prompt.

Before it edits, make it state:

current question
files it plans to touch
behavior it plans to change
things it intends to keep fixed

Afterward, make it state:

files actually changed
behavior actually changed
new dependency / schema / architecture decisions introduced
how to compare against HOME

If it says:

“To solve this properly I should also refactor the scene manager, rewrite the placement system and change the coordinate abstraction…”

that is not automatically wrong.

But it is a new branch of the decision, not permission to silently do all three.

I also like a small PARKED_IDEAS section:

PARKED
- possible scene-manager refactor
- replace collider representation
- investigate another LoD strategy

The idea is saved, so you do not feel forced to implement it immediately.

That is especially useful with AI because an assistant is very good at generating another plausible next step.

Your project needs a way to say:

“interesting; not this experiment.”

For the occasional change that really alters the shape of the project, I would leave one tiny decision note:

DECISION:
WHY NOW:
ALTERNATIVE I DID NOT CHOOSE:
REVISIT IF:

That is the basic idea of an Architecture Decision Record: preserve why an important choice was made so a future you — or a fresh AI — does not “fix” it back into an old problem.

And if your coding assistant supports repository instructions, put stable rules there rather than repeating them in every chat. GitHub Copilot, for example, supports repository/path-specific instructions.

Keep that entry point short. OpenAI reports the same pattern in Harness Engineering: a short AGENTS.md serves as a map to deeper project knowledge, not one giant instruction manual.

If the 'same room' may not actually be the same room

This is the 3D-specific part I would keep even if you never adopt much formal software process.

For this project, a “room version” is really something like:

Marble world / generation
        +
SPZ
        +
collider mesh
        +
coordinate/export convention
        +
scene transforms
        +
placement data
        +
code checkpoint
        +
Three.js / Spark / dependency versions

So I would keep a tiny scene manifest.

A spreadsheet is completely fine:

scene	Marble world/version	SPZ	collider	code checkpoint	transform note	status
gallery-03	…	g03.spz	g03.glb	home-03	…	HOME
gallery-04	…	g04.spz	g04.glb	home-03	…	untested

If an asset is regenerated, I would avoid silently overwriting the old file with the same name.

This is also one place where I would not trust a remembered coordinate rule.

World Labs’ current export specs and release notes show that coordinate/export behavior has changed and that export choice matters. The open-source SPZ format also has its own coordinate-system convention/metadata, and Spark’s loading guide shows explicit reorientation in some cases.

So instead of:

“Marble/SPZ always needs transform X”

I would record:

Marble export option:
SPZ convention / metadata:
collider convention:
loader conversion:
scene transform:

It is the luggage-tag idea: enough information to tell whether this really is the same bag.

Also remember that World Labs describes the collider mesh as coarse physics geometry rather than the visual representation. “The collider is correct” and “the splat is correct” are separate checks.

If a room works fresh, but breaks/slows after switching or reloading

Then I would hold the world/assets fixed and test runtime state separately.

A boring reproduction sequence is useful:

load HOME room
wait until splat is ready
record behavior / FPS / memory indicators

leave room
load another room

return HOME
record the same things

repeat

Do not regenerate Marble or rewrite placement during this test.

The question is only:

“Does something accumulate, survive, or initialize differently across transitions?”

Three.js resources often need explicit lifecycle handling; removing an object from a scene is not the same as disposing its geometry/material/resources. The official disposal guide also points to renderer.info as a useful inspection point.

Spark likewise has explicit loading/readiness and disposal state. Current SplatMesh exposes onLoad, initialized, isInitialized and dispose(), while SparkRenderer has its own dispose().

So I would distinguish:

not loaded yet
vs
loaded but wrong
vs
old state not cleaned up

before changing the scene architecture.

If yesterday's checkpoint changes after reinstalling/updating

Then freeze the dependency state before debugging the room.

I would record:

Node version
package.json
package-lock.json
Three.js version
Spark version
browser used for the baseline

npm’s package-lock.json records the resolved dependency tree and is intended to make later installs reproducible.

npm ci is useful for the question:

“Does this old checkpoint still reproduce from its lockfile?”

During a scene-debugging experiment, I would avoid upgrading Three.js, Spark, the build tool and the scene code simultaneously.

If you want to update dependencies, fine — but then the update is the experiment.

Same cake, same oven principle.

If you keep trying plausible fixes but are not learning anything

Then I would stop asking:

“What should I try next?”

and ask:

“What are the two or three possibilities I am currently unable to distinguish?”

Example:

OBSERVATION
painting looks too bright compared with the splat wall

POSSIBILITY A
Three.js color/material pipeline is mismatched

POSSIBILITY B
color pipeline is correct; baked room appearance simply needs local adaptation

CHEAP TEST
render a neutral reference through exactly the same Three.js pipeline

Or:

OBSERVATION
two rooms do not read as the same building

POSSIBILITY A
independent generation is re-imagining architecture

POSSIBILITY B
the rooms are acceptable, but scale/registration/transition makes them feel unrelated

CHEAP TEST
compare fixed views / dimensions before changing generation again

This is essentially the hypothesis → controlled test → observation loop in Google SRE’s troubleshooting chapter.

The experiment does not need to fix the museum.

It only needs to make you less uncertain than before.

---

I would keep the creative decisions flexible and make only the project state precise

I would not try to turn questions like these into fake metrics:

• does this room actually feel Renaissance?
• does the spotlight feel theatrical rather than artificial?
• does the transition feel architectural?
• does a normalized painting size look curatorially acceptable?

Those are design judgments.

What I would make precise is:

which room
which world
which SPZ
which collider
which code checkpoint
which experiment
which result

That gives you a nice middle ground:

creative judgment can remain subjective while the route to each candidate stays recoverable.

If you ask another person or AI for help, give them the map pin, not four weeks of footprints

A compact handoff is usually enough:

QUESTION:
HOME:
candidate:
expected:
actual:
shortest reproduction:
what changed:
what stayed fixed:
screenshot / golden view:
relevant error/log:
what I already tried:

That is the minimal reproducible example idea adapted to a visual/3D project — and a clean way to start a fresh AI conversation.

Instead of:

“Please understand this enormous chat and remember what happened three projects ago…”

you can say:

“Here is HOME. Here is the candidate. Here is the one thing that changed. Here is what I observed.”

---

Links I would keep around, but not try to learn all at once

For keeping your bearings

• GitHub Desktop — first repository
• GitHub Desktop — viewing history
• Git LFS
• Google SRE — effective troubleshooting
• Google SRE — incident response
• Remaining Oriented During Software Development Tasks
• OpenAI — Harness Engineering
• Minimal reproducible example

For this particular Marble / SPZ / Spark / Three.js stack

• Marble — Create & edit
• Marble — Chisel
• Marble — Expand
• Marble — Compose
• Marble — export specs
• Marble — release notes
• Marble — collider/mesh export
• SPZ repository/specification
• Spark — loading splats
• Spark — SplatMesh
• Spark — SparkRenderer
• Spark — splat editing
• Spark — performance
• Three.js — color management
• Three.js — disposing resources
• package-lock.json
• npm ci

Treat these as a shelf, not a syllabus: open the one that matches the problem in front of you.

So, if I had to answer “how do I start?” in one checklist

If I were in front of the project tomorrow, I would do this:

1. choose one working room
2. call it HOME
3. make one visible code checkpoint
4. write down the exact SPZ/collider/world that belongs to it
5. create PROJECT_STATE.md
6. choose ONE of the six original questions
7. define the smallest experiment that answers it
8. tell the AI what it must not change
9. run the same small checks / fixed views
10. KEEP IT, GO HOME, or PARK IT

Then repeat.

If that already stops the project from turning into final-final-really-final-27, I would not add more process.

And if it does not help, that is still a useful result: now the problem is more specific. It is probably time to open only the relevant detour — asset identity, AI change scope, runtime lifecycle, dependency drift, or an experiment that is not separating the possibilities.

That is much easier to work with than “the whole project no longer fits.”

[Difacto]

Hi John,

thank you again… so so much I read your complete answer very carefully, and the HOME / one-room laboratory approach makes a lot of sense to me. I’m going to use it.

I have one practical decision left before I start the first experiment.

Marble gives me the stronger atmosphere, but some generated rooms drift architecturally. A controlled Blender/Three.js test keeps the geometry stable, but currently looks too synthetic and plastic. The final experience must run in the browser, including Safari on iPhone and iPad, and the exhibitions need to remain easy to change every month.

May I ask three very concrete questions?

1. Have you personally tested SparkRenderer.renderEnvMap() and SplatEdit with INFINITE_CONE on a Marble-exported .spz? If yes, which Spark version and which browsers or devices did you test?

2. For fixed baked lighting, would you create and cache one local environment probe per artwork position, or is there a reason it needs to be generated again during interaction?

3. In this situation, would you lean towards a hybrid system — authored geometry/collider for stable architecture and interaction, with the splat as the visual layer — or would you avoid mixing both systems?

If you know of a minimal example showing a SplatEdit cone scoped to one room or one painting, including clean removal when changing rooms, even just a link would help me enormously.

I’m not asking you to solve the whole project for me. I would simply like to avoid beginning with the wrong architectural direction. Your replies have already helped me bring some order back into it. Thank you!

Diana

[John6666]

Well… I’m not very knowledgeable about Apple hardware or 3D​, but from what I could find and lightly test:

---

Short answers first

I think your HOME / one-room experiment can start with the hybrid architecture. I would not treat “splat vs authored geometry” as an either/or choice; I would give each representation a narrow job and keep the monthly exhibition layer independent of both.

For your three concrete questions:

1. Yes, I tested the current Spark path on a real public Marble-generated .spz after reading your question. I used Spark 2.1.0 + Three.js 0.180.0 and the public World Labs third-person starter’s attic.spz. In that test:

   • a room-scoped SplatEdit with INFINITE_CONE visibly changed the Marble splats;
   • removing the edit restored the captured baseline cleanly;
   • renderCubeMap({ update: true }) followed by readCubeTargets() produced useful Marble content on all six cube faces;
   • renderEnvMap() produced a nontrivial environment map which visibly lit an otherwise-unlit ordinary Three.js metal sphere.

   Important limitation: this was Headless Chromium 151.0.7922.34, WebGL2, on Linux. The actual GL renderer was SwiftShader, so this is a correctness/proof-of-path test, not GPU-performance evidence. I have not tested Safari, iPhone or iPad.

2. For fixed baked lighting, I would normally generate the probe once for a stable artwork slot/probe position and cache it. I would not regenerate it just because the visitor approaches the work. I would invalidate the probe when the room/capture state changes, not when the monthly artwork changes.

3. I would lean toward the hybrid. More specifically, I would keep the authored/collision side deliberately simple and let Marble remain responsible for the atmosphere. I would not try to make either representation be the source of truth for everything.

My default split would be:

Marble SPZ
  -> visible room / atmosphere / baked appearance

simple collider or authored proxy
  -> movement, blocking, coarse spatial structure

authored artwork slots
  -> exact hanging position, wall normal, width/height

ordinary Three.js artwork/frame meshes
  -> monthly data-driven exhibition content

cached local environment probes
  -> local IBL for inserted geometry

room-scoped SplatEdit
  -> the temporary spotlight/spill-like effect

That also gives you a useful escape hatch: if Marble’s visual result is good but one generated collider is awkward, you can replace only the operational geometry rather than rebuilding the visible room.

World Labs’ own current Third-Person Character Controller showcase is reassuring here: it explicitly combines a Marble-generated splat world, a paired collider mesh, Spark, Three.js and Rapier. The point is not that their starter proves your gallery architecture, but that this visual/physics representation split is already a normal enough way to work with Marble rather than an exotic workaround. The corresponding open-source starter is also useful to inspect.

The HOME I would build first

I would make the first proof almost aggressively small:

ONE ROOM
│
├─ one Marble SPZ
├─ one collider OR tiny authored collision shell
├─ one authored wall slot
├─ one Three.js painting/frame
├─ one cached local environment probe
└─ one room-scoped INFINITE_CONE edit

I would consider that proof successful if only these things work:

1. the room renders reliably;
2. the collision/operational layer is approximately registered with it;
3. one painting remains exactly where its authored slot says it belongs;
4. the cached environment probe makes the painting/frame sit acceptably in the baked room;
5. the cone visibly affects the intended splat region;
6. removing the cone restores the room cleanly.

Then I would expand to the second artwork, second room, monthly rotation, etc.

In other words, I would not try to validate all eleven scenes, all placement heuristics, mobile optimization and room streaming at once. If this small HOME works, each later failure has a much smaller search space.

What I actually tested on the Marble SPZ

The exact public Marble asset I used was the current attic.spz from the World Labs third-person starter.

Environment:

@sparkjsdev/spark  2.1.0
three               0.180.0 / r180
browser             Chromium 151.0.7922.34
API                 WebGL2
OS                  Linux
GL renderer         ANGLE -> Vulkan -> SwiftShader
Apple hardware      not tested
Safari              not tested
iPhone/iPad         not tested

Spark 2.1.0 is currently the latest published Spark release I found; the 2.1.0 release notes also specify Three.js 0.180.0 or newer as the peer dependency.

Cone

On the visible Marble room, enabling a child SplatEdit / INFINITE_CONE changed roughly 35–41% of pixels under the thresholds I used, so this was not just “the API call completed”; the room image visibly changed.

After:

roomSplat.remove(spillEdit);

the tested view returned pixel-for-pixel to the captured baseline.

That does not prove every Marble export or every LoD lifecycle case, but it answers the narrower question I had before: a real current Marble-generated SPZ can respond visibly to an INFINITE_CONE edit in Spark 2.1.0.

Spark’s own Splat Editing documentation explicitly describes INFINITE_CONE as one of the supported SDFs and says MULTIPLY or ADD_RGBA can be used for simple lighting-like effects; it specifically mentions infinite cones for spotlight-like regions.

Cube capture / environment map

I also tested the newer SparkRenderer cube/env-map APIs directly.

The decisive path was:

await spark.renderCubeMap({
  scene,
  worldCenter,
  update: true
});

const sixFaces = spark.readCubeTargets();

const envMap = await spark.renderEnvMap({
  scene,
  worldCenter,
  update: true
});

With Spark’s built-in update:true path, all six cube-target faces contained nontrivial Marble room imagery.

I then assigned the resulting envMap to an ordinary Three.js MeshStandardMaterial on a metal sphere with no other light. The rendered sphere changed substantially, so the result was not merely a non-null or empty texture.

That is consistent with Spark’s documented implementation: renderEnvMap() updates/sorts the splats around the capture point, renders the six cube directions, runs the result through Three.js PMREMGenerator, and returns a THREE.Texture suitable for MeshStandardMaterial.envMap.

There is also an official Spark environment-map example which follows essentially that pattern: render an environment map at the object’s position, optionally hide the object during capture, then assign the texture to its material.

One small result I would preserve because it surprised me:

manual six-view preparation
+ renderCubeMap(update:false)
        -> only 2/6 directly-read faces were useful

Spark built-in
renderCubeMap(update:true)
        -> 6/6 useful faces

So I would not start by trying to hand-roll a six-camera LoD prewarm. Spark’s built-in update path worked better in this test.

A later renderEnvMap(update:false) also produced a useful IBL result after more rendering had occurred, but I would not interpret that as proof that the initial incomplete two-face prepared state was sufficient; progressive LoD/page residency could have advanced meanwhile.

So for the first implementation, my boring default would simply be:

renderEnvMap({ ..., update: true })

generate once, then cache the resulting texture.

How I would cache the probes

For your fixed baked-lighting case, I think the useful distinction is:

STATIC APPEARANCE
room + fixed probe position
        ↓
renderEnvMap()
        ↓
cached texture


DYNAMIC INTERACTION
visitor approaches painting
        ↓
SplatEdit / INFINITE_CONE
        ↓
temporary splat colour change

I would not make the probe cache primarily an artwork-ID cache.

I would make it a capture-state cache, approximately:

roomVisualVersion
+ slotId
+ probePoseVersion
+ bakedLightingVersion
+ probeSettingsVersion

So:

January:
slot_03 -> Botticelli_A

February:
slot_03 -> Painting_B

March:
slot_03 -> Painting_C

can all reuse the same probe if:

same room/SPZ
same slot
same probe point
same baked room appearance
same capture settings

I would regenerate when something that the environment capture actually sees has changed, for example:

• a different room/SPZ revision;
• a moved slot/probe position;
• changed static splat edits;
• changed fixed geometry that should appear in the probe;
• different capture/PMREM settings.

I would not regenerate merely because:

• the visitor approaches;
• the overlay opens;
• the artwork image or metadata changes;
• the monthly exhibition rotates to a different work at the same slot.

Also, “one per artwork position” need not become a rigid rule. If the paintings are mostly matte and several slots on one wall look indistinguishable with one shared probe, one probe per wall/zone may be sufficient. Conversely, if a frame is metallic or two slots see very different surroundings, individual slot probes may be worthwhile.

So I would begin with the coarsest cache granularity that looks right and only split it when a visible mismatch appears.

This seems particularly attractive because renderEnvMap() is real work: splat update/sort, six renders, then PMREM filtering. There is little reason to put that work in the proximity-interaction loop when the captured room is static.

One other practical detail: I would generate the static probe with the dynamic cone/spill disabled. Otherwise the transient interaction effect becomes baked into the texture that you then reuse.

Why I would separate the hybrid into several ownership layers

The strongest reason I see for the hybrid is not “hybrid rendering is better”; it is that it lets failures remain local.

I would separate at least these three kinds of truth:

VISUAL TRUTH
"What does the room look like?"
-> Marble SPZ

OPERATIONAL / PHYSICAL TRUTH
"Where may the visitor walk or collide?"
-> collider / simple authored proxy

CURATORIAL TRUTH
"Exactly where may this artwork hang?"
-> authored slot metadata

The third one is important.

I would not make the low-resolution physics collider the permanent precision authority for your monthly hanging system unless it happens to be exceptionally clean.

Your current three-height wall analysis can still be useful as a one-time authoring/bootstrapping tool:

generated collider
      ↓
three-height / coplanarity analysis
      ↓
candidate usable wall regions
      ↓
human or automatic acceptance
      ↓
freeze stable slot anchors

After that, the monthly exhibition data only sees:

{
  "slot": "gallery03_wallB_02",
  "artwork": "work_184"
}

and the slot owns the permanent geometry:

position
rotation / wall normal
maximum width
maximum height
possibly probe ID
possibly interaction/viewing pose

That preserves the part of your current system that is already working while removing the noisy generated collider from the monthly workflow.

It also means the choice you discussed earlier — sacrificing exact physical artwork scale in favor of stable data-only hanging — remains a curatorial/content rule, not something coupled to Marble or the collider.

If the generated collider is bad

I would not interpret that as “hybrid failed”.

World Labs also has a Collider Builder specifically for placing simple box/sphere/cylinder collision shapes directly over a Gaussian-splat scene and exporting them as .glb.

So there is a cheap fallback:

keep:
  Marble visual room

replace:
  troublesome generated collider

author only:
  floor
  walls
  door openings
  important furniture/blockers
  interaction surfaces

For a gallery, the invisible operational geometry can be drastically simpler than the visible scene.

That seems preferable to rebuilding the whole room in Blender just to obtain stable collision and hanging anchors — particularly if the authored visual version is the part you currently find too synthetic.

Why I think this is a fairly safe direction to try

Again, I would not treat a showcase as an API contract, but the current World Labs Third-Person Character Controller already uses:

Marble splat
+
paired collider
+
Spark
+
Three.js
+
Rapier

and describes the splat world and collider as separately replaceable pieces.

I also loaded that exact public attic.spz + collider.glb pair together. With the starter’s published root transforms, they were coarsely registered as the same room: I did not see a gross 90-degree rotation, mirroring, order-of-magnitude scale mismatch or large translation mismatch.

I would only claim coarse registration for that public pair, though. I would not assume every historical Marble export, API result and Studio export shares one universal transform convention.

Internally I would keep separate transforms even if they happen to be numerically identical:

T_splat_to_app
T_collider_to_app
T_slots_to_app

That leaves room to correct one export path without disturbing the others.

Minimal room-scoped INFINITE_CONE

For your final “even just a link” question: the official Spark Splat Editing docs contain the key scoping rule.

A SplatEdit can be attached to a specific SplatMesh; if it has no SplatMesh ancestor, it applies globally to editable splats. So I would make the edit a child of the room it belongs to:

scene
└─ roomSplat
   └─ spillEdit
      └─ coneSdf

A minimal pattern is roughly:

import * as THREE from "three";
import {
  SplatEdit,
  SplatEditSdf,
  SplatEditSdfType,
  SplatEditRgbaBlendMode,
} from "@sparkjsdev/spark";

function createArtworkSpill(roomSplat, originLocal, targetLocal) {
  const spillEdit = new SplatEdit({
    rgbaBlendMode: SplatEditRgbaBlendMode.ADD_RGBA,
    softEdge: 0.15,
  });

  const cone = new SplatEditSdf({
    type: SplatEditSdfType.INFINITE_CONE,

    // For ADD_RGBA, keeping alpha at zero avoids making
    // low-opacity splats more opaque.
    opacity: 0,

    // Deliberately modest values; tune visually.
    color: new THREE.Color(0.10, 0.08, 0.055),

    // Spark documents the cone half-angle as:
    // pi/4 * radius.
    // radius = 0.5 -> about 22.5 degree half-angle.
    radius: 0.5,
  });

  cone.position.copy(originLocal);

  // Spark's cone is oriented along its local axis;
  // point local -Z from the artwork/light origin toward the wall.
  const direction = new THREE.Vector3()
    .subVectors(targetLocal, originLocal)
    .normalize();

  cone.quaternion.setFromUnitVectors(
    new THREE.Vector3(0, 0, -1),
    direction
  );

  spillEdit.add(cone);

  // This is the important scoping step:
  roomSplat.add(spillEdit);

  return spillEdit;
}

Then temporary interaction can be as simple as:

spillEdit.visible = true;   // visitor stops at artwork

spillEdit.visible = false;  // visitor leaves

and permanent teardown when changing/removing the effect:

roomSplat.remove(spillEdit);

In my small generic test, visible=false restored the baseline. More importantly, on the exact public Marble LoD room, detaching the edit restored the captured baseline exactly in the tested view.

So if I wanted the least surprising lifecycle, I would use visibility for short on/off interaction and actual detach/removal when the room/effect is being destroyed.

The official dynamic-lighting example is also useful even though it uses spherical SDF lights rather than your cone: it shows the intended SplatEdit + ADD_RGBA pattern for lighting-like splat colour changes.

One caveat from the docs: ADD_RGBA can become hyper-saturated, and non-zero added opacity can make low-opacity splats more opaque. That is why I would start with small RGB additions and opacity: 0, then tune the effect visually rather than trying to make it behave numerically like a physical Three.js SpotLight.

The cone is ultimately a spatial colour edit on baked splats, not a recovered physical light.

The failure tree I would keep for later, rather than solving everything now

I think this is the useful part of the architecture: if HOME fails, the failure tells you which representation needs attention.

A. Public/known-good Marble SPZ works,
   but your own exported SPZ does not
        |
        v
   export / provenance / transform gate
   rather than immediately blaming the whole Spark architecture


B. Room + collider broadly agree,
   but the painting will not sit exactly on the wall
        |
        v
   placement-authority gate
   -> freeze authored artwork slots
   -> do not ask coarse collider triangles to be precision anchors


C. Painting position is correct,
   but a splat doorway / foreground region draws in the wrong order
        |
        v
   depth / occlusion gate


D. Desktop HOME works,
   but Safari/iPhone/iPad does not
        |
        v
   Apple/WebKit/device gate


E. One room works,
   but room A -> B -> A produces stale content or memory growth
        |
        v
   LoD / paging / disposal lifecycle gate

C. Depth / occlusion

I would not solve this pre-emptively, but it is worth knowing where the switch lives.

The current SparkRenderer documentation exposes both:

depthTest
depthWrite

and says splats can respect opaque Three.js Z-depth with depthTest.

It also warns that depthWrite may give undesirable results because much of a Gaussian splat is transparent.

So if the painting looks correct head-on but fails when a doorway, column, chair, etc. should pass in front of it, I would open this gate then. A single artwork viewed from a few oblique camera positions should reveal the problem very cheaply.

I would not treat an occlusion problem as evidence that all hybrid composition is wrong.

D. Safari / iPhone / iPad

This is the largest thing I cannot answer empirically.

Because those are actual target devices for you, I would make one real-device smoke test an early gate, but only after the tiny HOME is otherwise correct. Before expanding to eleven scenes, I would want to know that one representative room can:

load
walk
display one artwork
apply/remove one SplatEdit
reuse one env map
survive a few minutes

on actual Safari/iPhone/iPad.

The current Spark renderer docs already distinguish mobile and desktop LoD budgets. For example, their defaults target fewer splats on mobile, and maxPagedSplats has a specifically lower iOS default than desktop. That is a good reminder that the desktop correctness test does not settle memory/frame-time behavior on iOS.

There have also been platform-specific upstream reports. For example, Spark issue #399 described a Safari shader-link failure on one macOS 27 beta and explicitly identified it as a Safari/ANGLE/Metal beta regression rather than a general Spark failure; that issue is now closed.

I would use reports like that only as evidence for the failure axis:

browser/device-specific behavior exists

not as evidence that your project has that bug.

E. Room lifecycle / LoD

Similarly, I would keep “remove the cone” separate from “unload the entire LoD room”.

The cone itself looks straightforward: detaching its room-child edit worked cleanly in my test.

Whole-room streaming/paging is a larger lifecycle problem. If you later move to chunked .rad or aggressive multi-room streaming, I would specifically test:

A -> B -> A -> B -> A

and watch:

memory
network requests
old edits
old probes
old renderer resources
stale chunks

rather than assuming a one-room cleanup result covers it.

For context only, there is currently an open Spark report, #384, about queued chunks from chunked RAD scenes surviving a scene change. That is not evidence of a bug in your .spz project; it is just a good example of why I would make multi-room streaming its own gate if/when you need it.

So, if I had to choose the architecture before writing much more code

I would start here:

                    HOME

              Marble SPZ
             /          \
            /            \
   visible atmosphere   room-scoped SplatEdit
                              |
                              |
                    temporary interaction spill


 simple collider / authored shell
              |
              |
      walking / blocking


      authored wall slots
              |
              |
 exact exhibition placement


 Three.js painting + frame
              |
              |
 monthly data-only rotation


     cached local env probe
              |
              |
 appearance integration

The main thing I would avoid is making one representation carry responsibilities it is bad at:

• do not make the splat your precision interaction geometry;
• do not make a coarse collider your permanent curatorial hanging database;
• do not rebuild environment probes for an interaction that does not alter the baked room;
• do not rebuild the whole room in Blender merely because one operational layer needs more stability.

That leaves you with something fairly reversible.

If Marble keeps winning visually, you keep it.

If the generated collider disappoints, you replace only the collider.

If automatic wall analysis is good enough, use it to bootstrap slots; if not, author a few slots once.

If a monthly exhibition changes, the room architecture does not know or care.

And if Safari exposes a problem, you have a small enough HOME that you can tell whether the failing layer is Spark/LoD, composition, memory, or the application around it instead of debugging the whole eleven-scene project at once.

So I don’t think the hybrid direction looks like the “wrong architectural direction” from the evidence I could get. I would treat it as the leading first prototype, with Apple/mobile, depth/occlusion, and full room-lifecycle behavior kept as separate gates rather than reasons to block the experiment before it starts.

[Difacto]

Thank you soooo much my ai helper loves your help ))))too