STEP loader antlr v4 based

[rimmartin]

Hi,

I made an antl4 v4 grammar to create an initial parsing/interpreting layer https://github.com/antlr/antlr4-javascript.

This does introduce some runtime dependency on antlr4. Would this preclude proposing such a loader someday?

[mrdoob]

Hmm... I'm failing to see how does this relate to three.js...?

[rimmartin]

I see other loaders? No interested in STEP format?

[mrdoob]

Ah wait. I didn't know STEP was a format. I just looked at the antlr4 stuff.
Antlr4 seems to bit quite a dependency...

[rimmartin]

STEP/Brep/IGES are produced by Open Cascade based CAD tools and some used by NC machine tools. I was exploring parsing it and populating a threejs visual.
Examples:
http://www.steptools.com/support/stdev_docs/stpfiles/ap214/index.html
In the data section each line starts with an 'instance' #xxxx. These get referenced by other instances so while making a visual the access is by these references into instances. In other words not a top down format. Instead of making a human based parser I develop a grammar for STEP which generates a lexer/parser that is more accurate/validating and fast. Albeit at run time there are some dependencies

                             var antlr4 = require('antlr4/index.js');
                            var STEPLexer = require('lib/parsers/grammars/STEPLexer.js');
                            var STEPParser = require('lib/parsers/grammars/STEPParser.js');
                            var STEPClientListener = require('lib/parsers/grammars/STEPClientListener.js');
                            var chars = new antlr4.InputStream(data);
                            var lexer = new STEPLexer.STEPLexer(chars);
                            var tokens  = new antlr4.CommonTokenStream(lexer);
                            var parser = new STEPParser.STEPParser(tokens);
                            parser.buildParseTrees = true;
                            var tree = parser.step_translator();
                              var actualSTEP = new STEPClientListener();

                            antlr4.tree.ParseTreeWalker.DEFAULT.walk(actualSTEP, tree);
                            buildUpSTEPVisual(actualSTEP);

I'll explore some more if parsing code can be written by hand.

Currently I'm looking at the translation to threejs. threejs, like working with oce(c++ open source library version of opencascade) starts say an extrusion with a 2d face and then sweeps it to 3d. STEP is 3d for its advanced_face's that make up a part's surface. Either have to use heuristics to get back to 2d or develop vertice/face operations in 3d. In STEP also holes drilled are in 3d. Begin to sense this doesn't remain as lightweight as I had hoped when I started exploring.

http://www.steptools.com/support/stdev_docs/express/ap232/html/schema.html

[Usnul]

Maybe of interest, but there is a fairly light-weight parser generator for JS that has no dependencies for generated code:
http://pegjs.org/
Downside is that it is a lot more restricted than antlr meaning grammar definition becomes a bit more of a pain.

[rimmartin]

thank you @Usnul I'll see if the pegjs grammar is doable for this format

[bhouston]

@rimmartin, https://Clara.io imports STEP and IGES btw and can export to Three.JS -- and you can use it all via the REST API if you just want to use it for conversion purposes. http://clara.io/learn/sdk/api

[kylegmaxwell]

+1 for interest in STEP format loader.

[charliestirk]

STEP is complex, and in most cases will need to be processed by a CAD kernel to get to a format that three.js can currently load. The reason is that STEP contains several representations for geometric shapes, the most common being Advanced BREP (often with NURBS and trimming). There is an open source project that generates a low-level API for STEP data (www.stepcode.org) that is used by BRL-CAD and OpenVSP. OpenCasCade has its own STEP translation capability, which is used by FreeCAD (and others) for STEP and IfcOpenShell for IFC. Clara.io can import both STEP and IFC, but like many other free tools (including open source such as FreeCAD and OpenCasCADE) lacks modern STEP capabilities like associative 3D text, supplemental geometry, model styling and organization, tessellation, user defined attributes, PMI, etc. (www.cax-if.org). The NIST STEP File Analyzer can convert most modern STEP capabilities to VRML/X3D, and is free but not open source. So unless three.js develops capabilities to cover more of the STEP geometric representations, STEP and IFC data will need to be processed by something into a form that three.js can load (like three.js json, STL, VRML/X3D, COLLADA, etc.)

[bhouston]

@charliestirk Clara.io uses the OpenCascade kernel to import STEP. :) We recently added support for instancing, LOD generation in tesselation, and hierarchy extraction (this last one goes live next week) when importing STEP files.

[rimmartin]

At the start I mentioned an antl4 v4 grammar and a layer that I have putting the STEP 'description' in memory. Then was looking to go to threejs constructs but soon learned to do everything I wanted would make threejs much heavier; adding not only the runtime of antlr but also require a lot of new vertex processing. This I believe would be the wrong direction for threejs which is doing a great job for its purpose.

What I didn't describe was the goal I am trying to achieve. Inspired by node-occ https://github.com/erossignon/node-occ exploration in making a javascript scripting technque for building up a 3d model, after running into severe limitations I started a new koa based app that uses the ace editor https://ace.c9.io/#nav=about and uses c++ code based on OpenCascade as a 'script' and when valid the server side compiles and produces a STEP. Then I want to display(without translators) the results back to the HUD I was creating with the koa app.

[rimmartin]

I was viewing the intermediate STEPs in Salome http://www.salome-platform.org/ but there isn't a single version I've tried that doesn't freeze on my computers. So was finding a better way

[charliestirk]

@bhouston Good to know that Clario.io is using OpenCasCade to import STEP. Do you also use IfcOpenShell for IFC? Your contributions to three.js are well known, and I agree that the Clara.io API is a good path from STEP to three.js. For STEP hierarchy extraction, I assume you mean what STEP calls assembly structure? It would be useful if Clara.io could handle some of the modern CAX-IF Recommended Practices. Regarding the OpenCasCade STEP translator, there has been discussion in the OCE, OCCT, IfcOpenShell and STEPcode communities about upgrading it to handle recent STEP capabilities, but a switchover to STEPcode would be a huge task. Besides new capabilities, if there were memory or speed improvements, then it might be worth it. In the meantime, the OCCT developers have been good about fixing bugs in the current translator.

[bhouston]

@charliestirk I do not believe we import IFC well at this point, but it is on our list of things to do.

We do handle the assemblies from STEP. I am not yet familiar with CAX-IF Recommended Practices, rather we just read what OpenCascade can send us.

BTW you can use Clara.io via our node-clara Node.JS module from either Node.JS directly or from the command line -- import, export, convert scenes, render thumbnails in WebGL or V-Ray, and even apply editing operations, etc. This allows for fairly easy integration into your applications, and one of the export file formats from Clara.io is ThreeJS. :)

https://www.npmjs.com/package/clara

[morz]

+1 for interest in STEP format loader.

This comment added 6 Oct 2015. Now is 2017 year. I have a question: We have STEP Loader for ThreeJS? =)

[rimmartin]

I haven't pursued it because I'm a grammar parser developer and could see things I was thinking would be too bulky for this project which is really good at what it was designed to do; I can't see to alter it to get it to be an engine for CAD/CAM. There would be a lot of mesh building code to bridge in STEP because otherwise too much heuristics would be needed to get to the threejs geometry objects

https://threejs.org/docs/ lists
BabylonLoader
ColladaLoader
GLTF2Loader
MTLLoader
OBJLoader
OBJLoader2
WWOBJLoader2
PCDLoader
PDBLoader
SVGLoader
TGALoader

[alarvy7]

Is there any loader for STEP file in threejs. Actually I have used threejs to make a simple CAD product (www.alarvy.com). We want the capability to export 3D models created from our modeler to STEP file format. STEP format is the international standard for 3D model data exchange. This format is supported widely by all CAD/CAM softwares like SolidWorks, Catia, NX, Creo, Inventor, etc. I request you to please add this to the feature request. This is quite urgent for me. Is there any way to expedite this. I am ready to pay a donation of $250. I know it is opensource. But still I thought of adding this if any freelancer is interested to do this.

[mrdoob]

@alarvy7 Is there a spec of the format anywhere? We'll also need a bunch of example files.

[alarvy7]

I created these documentation around 2007.
http://trkannan.wikifoundry.com/page/Standard+for+Exchange+of+Product+model+data+%28STEP%29
http://trkannan.wikifoundry.com/page/STEP+AP-203
http://trkannan.wikifoundry.com/page/STEP+AP-203+FILE+FORMAT
I have attached lot of models in this link. These models were created in ProE 2000 (very old version).
https://drive.google.com/open?id=0B6pWSLDwBUdybk5OWU5XcUtSWEk . I will try to get more latest models. My email id is rnd@alarvy.com. If you need more info, you can ping me personally

[rimmartin]

@alarvy7 exporting STEP is a much more doable endeavor than loading STEP. There would be something in STEP for every mesh in threejs

[alarvy7]

@rimmartin . Yes I too felt that exporting will be easier than loading because STEP uses BRep format, where they define a solid in terms of bounding(enclosing) surfaces. Even exporting will be good enough for now. But just for your reference for importing STEP files, any solid would comprise of max 6 types of surfaces, planar, cylindrical, conical, spherical, toroidal and BSpline surface. We can very well create these first 5 surfaces using threejs geometries. For B_Spline surface probably we may use ParametricGeometry for now. I found the following link very helpful to understand NURBS and BSpline surfaces http://geometrie.foretnik.net/files/NURBS-en.swf . I will try to get few examples with BSpline surfaces.

[rimmartin]

@alarvy7 Looking around(googling) what I mean by heuristics is called by some researchers "feature extraction" from neutral file format STEP and these head in two ways; back toward CAD geometries and to what is needed to CAM from stock to the 'features' the designer intended. An example paper https://core.ac.uk/download/pdf/11783857.pdf which is by no means comprehensive. There is a whole field of study on this. I'm betting many manufacturers still have a human decide on the cam steps to get to the desired cad results

Most STEP I come across is ADVANCED_FACE'd (planar, cylindrical, conical, spherical, toroidal primitives and BSpline surface are rare) by the authors putting out the 'mesh'; not the geometries and can't retain( since it is neutral) the merging operations that occurred along the way to get to the mesh.

To me this step is so immense it is a separate operation from loading for viewing/editing in a threejs interface but rather predone and cached in the threejs format

I'm parsing STEP just fine. But am looking at writing STEP or directly to g code

[alarvy7]

@rimmartin My explanation was for simply loading the STEP file in threejs and creating the 3D model. Feature recognition/extraction is a very vast area and it will be domain specific. I have made few publications in Feature recognition. If you are interested you can get them form this link https://www.researchgate.net/profile/Kannan_Tr2/publications

[alarvy7]

@mrdoob I have attached latest STEP files in this link. https://drive.google.com/open?id=0B6pWSLDwBUdyTUduZnI0aHVuX2c . We can get more from GrabCAD as well. https://grabcad.com/library/bottle-opener-with-collection-bin-1
https://grabcad.com/library/water-rocket-set-1
https://grabcad.com/library/design-of-a-planetary-rover-with-an-on-board-robotic-arm-1/details?folder_id=3112781

[rimmartin]

The links I posted above to the spec are now behind an account login. There is the official https://www.iso.org/standard/63141.html where they want money to be in the club. Very little is open information

[kriffe]

If you are looking for validation models there are some to be found on: https://www.cax-if.org/library/index.html

Also just published a fork of wikistep.org that seems to be not beeing maintained any more on: http://wikistep.skymaker.se

[DEKHTIARJonathan]

I don't want to make any self promotion, however this can help some of you.
For a research project, I developed the website : https://www.dmu-net.org.

It has a visualizer for 3D Model which came in STEP format at the beginning: https://www.dmu-net.org/explore.php

In order to enable the display for 3D STEP models, I used OpenOCC to convert from STEP to ThreeJS.

The source code of this project is available here:

• Website: https://github.com/DEKHTIARJonathan/dmu-net.org
• STEP to ThreeJS Converter: https://github.com/DEKHTIARJonathan/dmu-net.org/tree/STEP-2-ThreeJS-BatchConverter

I hope it will be useful to some of you.

Best Regards,

Jonathan

[donmccurdy]

Also worth considering, cqparts can convert STEP to glTF: KhronosGroup/glTF#1325

[mrdoob]

Seems like converting to glTF does the trick for now. We can revisit in the future if someone comes up with a limitation of that approach.

[alarvy7]

How to export threejs to STEP format. glTF only provides option to convert STEP to threejs.

[alarvy7]

Yes, there is a STEP exporter too in gltf. Thanks

[theChumpus]

Has anybody been able to use cqparts to covert a STEP file to glTF and then import into a three scene?

I'm converting the bottle opener example (from above) to embdedded glTF but upon importing into the editor, I get an empty mesh and a shader compilation error:

THREE.WebGLProgram: shader error:  0 gl.VALIDATE_STATUS false gl.getProgramInfoLog invalid shaders ERROR: 0:988: '<' : wrong operand types - no operation '<' exists that takes a left-hand operand of type 'highp float' and a right operand of type 'const int' (or there is no acceptable conversion)

The mesh seems to exist (although it only seems to have position attributes, no normal or uvs).

NOTE I just checked and I can use cqparts to convert to JSON instead and that imports ok, even though the conversion only converts one child of the STEP file (I think this relates to cqparts/cqparts#64)

from cqparts import Part
Part.importer('step')('bottle_opener_1_0_asm.stp').exporter('json')(filename='bottle.json')

[fragmuffin]

@theChumpus
cqparts doesn't support normals, or mapping to textures (ie: uv indexes) yet.
Mainly because underlying library used (freecad) doesn't supply this when getting an object's mesh.
There is a push to port cadquery (the basis of cqparts) to use OCC directly instead of relying on freecad libraries, I'm sure there will be increased functionality options when that goes into alpha.

cqparts/cqparts#64 should fix the issue of multi-part STEP files only exporting the first object to glTF. But it won't add normals or uv's, that's an independent issue.

cqparts/cqparts#36 should greatly help with overall web-rendering support.

note: I'll be favoring glTF encoding over the json... json may be left to die... poor json.

[alarvy7]

@fragmuffin
I want to export threejs models to STEP format. We are creating 3d models in AlarvyCAD (http://www.alarvy.com/ ) using threejs. I want to export these models as STEP files.
There are two requirements discussed in this thread

1. Converting Threejs to STEP - Currently your STEP exporter expects shapes created in FreeCAD and so we are not able to create STEP files using Threejs. If there is any way we can create STEP file using cqparts, please let me know.
2. Converting STEP files to Threejs - Most of the above discussions are on the above issue.

@mrdoob We should not close this issue. Also we need a direct translator as you know there could be errors in intermediate translations. Mechanical CAD world uses STEP files a lot.

[fragmuffin]

@alarvy7 sorry, no, the only import type cadquery (and by extension: cqparts) has is for STEP format, so it cannot import the ThreeJS JSON format.

[theChumpus]

@fragmuffin Thanks for clarifying! Sounds great, I'll keep an eye on those issues.

Regarding the shader compliation issue I was having after importing the converted glTF; would it be the lack of normals/uvs that's causing this?

[donmccurdy]

@theChumpus are you able to share a glTF file that reproduces the compilation error? glTF does not require normals or UVs, so that should be fine.

[theChumpus]

@donmccurdy sure thing. Please find here: https://drive.google.com/open?id=1sfCgHWAIGHJfEAnj1mWbWP3i860JX2OT

I converted this https://grabcad.com/library/screw-141 using:

from cqparts import Part
Part.importer('step')('screw.STEP').exporter('gltf')(filename='screw.gltf', embed=True)
Part.importer('step')('screw.STEP').exporter('json')(filename='screw.json')

I've included both in the zip file; glTF does't work, json does.

[donmccurdy]

Thanks — still looking into this but the quick fix, if you want it, would be:

gltfObject.traverse((node) => {
  if (node.isMesh && node.material.alphaTest === 1.0) {
    node.material.alphaTest = 0.999;
    node.material.needsUpdate = true;
  }
});

That shouldn't be necessary of course; filed #13976 to track.

[theChumpus]

@donmccurdy Awesome, thank you so much!

[kovacsv]

Pretty old thread, but maybe someone is interested in my progress on the topic. Basically the STEP format is everything but web-friendly. It's hard to parse, hard to generate the triangulation from the mathematical representation, and you need an enormous size of library to do the job.

But, if for some reason you still want to parse STEP files on the client side, take a look at occt-import-js. It is based on OpenCascade, and I've tried to minimize the size of the wasm file, but it's still 15 megabytes.

The result of the conversion is a three.js compatible JSON, so I think it's pretty easy to write a loader using this library.