Cut a scene into named parts

An avocado has three texture islands and each one is a part. A furnished room is one mesh with 538 islands, most of them a few triangles of trim or the label on a prop, and no island is "the bed". SEG handles the part of the job that needs no judgment: it joins the islands that touch into components, measures them in the world frame, places everything you did not name on the nearest thing you did, and checks the result. What is left for you, or for a model, is the one step that never reduced to a rule: naming the anchors. Built with the JS Shell.

1 Same tool, different evidence

The object tutorial's evidence was mostly colour: the cut face of the avocado is cream and orange, the beak of the duck is orange. A room has almost none to give. Its texture is a dark green-black atlas shared by a few hundred pieces, and the two lines that matter most for telling a ceiling from a floor are not colours at all: they are where a piece is and how big. The tool is the same; the evidence that carries the weight is different, and the report changes with it.

Who chooses the grain. The host does, from the mesh: when there are at most three components for every four islands (the room has 538 islands and 282 components), SEG.describe and SEG.data report components; otherwise islands. A caller can say which: SEG.describe('#Mesh', 3, { grain: 'islands' }). The choice is stated in the last line of the report, and the grain the plan was written against is recorded.

2 The model, and its licence

Cyberpunk micro-apartments by Mikita_Hubanau, from Sketchfab, licensed Creative Commons Attribution-NonCommercial. Download the glTF and import the .glb with the dialog's defaults. The file has three meshes: Object_4 (12,151 triangles, textured, the room), Object_5 (7,794 triangles, one flat material) and Object_6 (a two-triangle glass pane). This page cuts Object_4.

3 Read the components

SEG.describe('#Object_4')

component 0: 2270 triangles (12 islands: 0,5,7,9,12,13,…), size 0.754,1.489,0.258, centre -0.919,2.989,-2.331, at 0.41,0.50,0.23, colours rgb(14,17,14) 72%  rgb(25,30,24) 18%  rgb(43,52,40) 10%
component 1: 1892 triangles (13 islands: 1,2,3,18,19,62,…), size 0.285,0.698,1.159, centre 0.340,1.362,-1.106, at 0.53,0.23,0.37, colours rgb(63,68,65) 48%  rgb(107,120,114) 37%  rgb(171,198,191) 15%
…
component 21: 118 triangles (4 islands: 36,84,123,251), size 5.694,0.471,7.736, centre 1.660,5.691,0.475, at 0.67,0.96,0.55, colours rgb(197,240,225) 59%  rgb(55,80,71) 21%  rgb(132,170,156) 20%
component 29: 75 triangles (11 islands: 122,186,245,246,247,248,…), size 9.193,1.007,7.028, centre 0.355,0.522,0.769, at 0.54,0.09,0.59, colours rgb(14,18,14) 42%  rgb(28,36,27) 40%  rgb(48,61,48) 18%
…
(538 islands joined into 282 components where they touch in space; world frame, y up. Pass { grain: 'islands' } for the islands. Name anchors: { name, classes, component: [..] }, and let SEG.attach place the rest)

282 lines instead of 538, and each one says what a person asks first: how big, where, how detailed. Component 21 is 118 triangles, 5.7 × 0.47 × 7.7, with at.y 0.96: a thin slab at the very top, a ceiling. Component 29 is 75 triangles, 9.2 × 1.0 × 7.0, at.y 0.09: a slab at the bottom, the floor deck. Component 1 is 1,892 triangles, 0.29 × 0.70 × 1.16, low and in the middle: far too detailed and too small to be furniture. Its colours are grey and white. A cat.

The same numbers as objects, for a script or a model: SEG.components('#Object_4'). Each has component, islands, triangles, size, centre, at and colours.

4 Name the anchors

This is the step that does not reduce to a rule, and the only one that is yours. An anchor is a component you can name from the evidence. Start with the biggest and most distinctive; the rest will follow them. Say what each name rests on: position, size, colour, or shape when all you can honestly offer is a description of the form. It is recorded, and a record with many shape names is a different claim from one with none.

The last row is the honest one. Nothing in the evidence says what that unit is, so the name describes what can be seen and the basis says shape. It is better than a confident Fridge.

Before cutting, ask where everything else would go:

SEG.attach('#Object_4', [
  { name: 'Floor_Deck',    classes: ['floor'],   component: 29 },
  { name: 'Cyber_Cat',     classes: ['cat'],     component: 1 },
  { name: 'Ceiling_Panel', classes: ['ceiling'], component: 21 } ], { maxDistance: 0.5 })

It returns parts (each anchor with what would be attached), unattached (what is too far), tie_broken, the parameters used and, per part, how many components it would take and how far the furthest was. Nothing is cut.

5 What attaching does, exactly

The rules are fixed and written down, so a run can be repeated and a result can be questioned.

1. Distance is to triangles, not vertices. A floor is a few huge triangles, and its four corner vertices are metres from anything standing on it. Each unnamed component is sampled at the centres of up to six triangles per island, and the distance is the shortest from a sample to any triangle of the anchor.
2. The nearest anchor wins, but near-ties go to the smaller thing. Every anchor within tie (0.05 world units) of the nearest is tied with it, and the one with the smaller bounding box wins, so a mug that touches both the counter and the floor belongs to the counter. A prop beats the structure it stands against. On the room the rule decided seven components, each given to a smaller anchor than the strictly nearest one. tie is a recorded parameter, in world units: change it if your scene is a hundred times bigger.
3. Beyond maxDistance, nothing is attached. There is always a nearest anchor, so without a limit a lamp standing alone across the room would join the nearest wall, and a component that is really an object of its own would be absorbed without a word. The limit defaults to a tenth of the scene's diagonal (1.448 on this room, which is 14.48 across) and the value used is recorded. A component beyond it is returned to you, named with its nearest anchor and the distance, and the cut refuses to run until you name it as an anchor of its own, or describe it the way Dark_Console_Unit is described, or raise the limit.

SEG.split('#Object_4', { group: 'Cyber_Room_Parts', source: { … }, anchors: [ … ], attach: { maxDistance: 0.3 } })
// with the first plan's anchors:
// SEG.split: 5 components are further than maxDistance (0.3) from every anchor: component 272 (nearest "Kitchen_Counter", 0.38659),
// component 273 (nearest "Bed_Frame", 0.55048), component 274 (nearest "Bed_Frame", 0.498), component 275 (nearest "Wall_Front_Right_Column", 0.56101),
// component 281 (…). Name each as an anchor of its own (component: [n]) or raise attach.maxDistance. Nothing was changed

6 Cut

The plan is the anchors, their classes, what each name rests on, and the source. That is all a model has to write.

SEG.split('#Object_4', {
  group: 'Cyber_Room_Parts', classes: ['room', 'interior'],
  source: { url: 'https://sketchfab.com/3d-models/cyberpunk-micro-apartments-fd8011e2af0e43288f5445600197ea77',
            fetchable: false,                      // a model page behind a login: the file cannot be re-fetched, so its hash and size are what identify it
            sha256: 'b2026982a0cd825a2046e2e1c5dc49ad5af56ad36e7aba1954118b12ce85ec73', bytes: 2266956,
            licence: 'CC Attribution-NonCommercial (as stated on the Sketchfab model page)',
            attribution: 'Mikita_Hubanau (Sketchfab: cyberpunk micro-apartments)' },
  anchors: [
    { name: 'Cyber_Cat',               classes: ['animal', 'cat'],               component: 1,  basis: ['position', 'size'] },
    { name: 'Bed_Pillow',              classes: ['bedding', 'pillow'],           component: 2,  basis: ['position', 'size', 'colour'] },
    { name: 'Doorway_Frame_And_Trim',  classes: ['structure', 'doorway', 'trim'], component: [7, 47, 68], basis: ['position', 'size'] },
    { name: 'Ceiling_Panel',           classes: ['structure', 'ceiling'],        component: 21, basis: ['position', 'size'] },
    { name: 'Floor_Deck',              classes: ['structure', 'floor'],          component: 29, basis: ['position', 'size'] },
    // … Wall_Left_Pillar 14, Wall_Right_Unit 19, Wall_Front_Right_Column 28, Bed_Frame 16, Crate_Stack [3,5,8,9,10,12,15,17],
    //   Plant_Cluster 23, Kitchen_Counter 22, Blue_Mug 4 …
    { name: 'Dark_Console_Unit',       classes: ['appliance', 'console'],        component: 0,  basis: ['shape'] } ] })

Cyber_Cat: 1892 triangles | Bed_Pillow: 643 triangles | Doorway_Frame_And_Trim: 350 triangles | Ceiling_Panel: 140 triangles | Floor_Deck: 95 triangles | Wall_Left_Pillar: 251 triangles | Wall_Right_Unit: 496 triangles | Wall_Front_Right_Column: 89 triangles | Bed_Frame: 428 triangles | Crate_Stack: 2367 triangles | Plant_Cluster: 1130 triangles | Kitchen_Counter: 1340 triangles | Dark_Console_Unit: 2590 triangles | Blue_Mug: 340 triangles
⚠ orphan sink: Plant_Cluster: it received 167 of the 259 components that were attached (64%): it is the nearest anchor to everything that has no anchor of its own. Name more anchors

The triangles add up to the mesh's 12,151, the parts share the original material so the room looks as it did, and one Undo puts Object_4 back. Then check it as before: SEG.preview('#Cyber_Room_Parts') and isolate('#Crate_Stack'), from every side.

7 The checks, and the cut they caught

The first cut of this room was wrong in a way the picture did not show. The thin brown trim that runs round the whole room was attached to the nearest anchor, which was Wall_Front_Right_Column, and the column came out room-sized. What gave it away was a number, not the picture: the part's box spanned 70%, 92% and 99% of the room on x, y and z while it carried 14% of the surface. That is now an assertion, and it fires on exactly that plan.

A part that declares itself spread out (a class of trim, frame, cable, pipe, wire, rail, network, edge or outline) is exempt from oversize and engulfing, which is how the fix is expressed: name the room-wide trim as one part and say so, as in step 6. Fewer than four parts are not judged. All the thresholds are in ASSERTIONS in meshSegment.js and are recorded with every cut.

A fired check does not stop the cut and does not change it. The cut is one undo step, and the right response to a failed check, undo and cut again with a better plan, is a decision for whoever wrote the plan. The check says which assertion, which part, and the numbers.

8 What the record keeps

const rec = $S('#Cyber_Room_Parts').nodes[0].userData.segmentation
rec.grain                              // 'components'
rec.plan                               // the anchors, names, classes, basis, source: as given
rec.describe.components                // the 282-line report the decision was made from
rec.attach.parameters                  // { max_distance: 1.448, max_distance_default: true, tie: 0.05, samples_per_island: 6, weld_digits: 4, scene_diagonal: 14.48, frame: 'world' }
rec.attach.per_part                    // per part: components, how many were attached, and the median, 90th percentile and maximum distance
rec.attach.tie_broken                  // the seven components the tie-break gave to a smaller anchor than the nearest
rec.attach.unattached                  // [] : nothing was left over
rec.signals.components_per_part        // Plant_Cluster 168, Crate_Stack 24, Cyber_Cat 1, …: a part made of 168 components is a different claim from one made of 1
rec.signals.names_by_basis             // { evidence: 13, shape: 1, unstated: 0 }
rec.signals.assertions                 // what fired, with the numbers; rec.signals.assertion_parameters says what the thresholds were

SEG.replay('#Object_4', rec) recomputes the components and the attachment from the recorded plan and parameters, and refuses to run if the mesh is not the one the record describes (different triangle counts, or different islands in a component). Run on a fresh import, it gave the same 14 parts, triangle for triangle, and the same attachment record.

9 Your own scene

1. SEG.describe('#mesh'). If it reports islands, the mesh is an object: use the object tutorial. If it reports components, read size, centre and at first.
2. Name the 8 to 20 components that you can justify, biggest first. Give each a basis. Use shape for what you can only describe.
3. SEG.attach('#mesh', anchors) to see where everything goes and what is left over. Name what is left, or raise maxDistance if you mean it.
4. SEG.split. Read every ⚠. Declare trim as trim. If a part took most of what was attached, name more anchors.
5. SEG.preview and isolate from every side; then lint() and requireLabels(true).