Org.simantics.scenegraph

org.simantics.scenegraph (SVN) is a scene graph library for 2D rendering with a focus on supporting remote rendering using a client-server architecture.

Overview

org.simantics.scenegraph provides a generic serializable scene graph datastructure ( wikipedia:Scene graph ) for displaying 2D and 3D graphics.

The idea of scene graph is that it is an independent datastructure that can be rendered as such. It can only access local variables, thus it provides a strict interface between the visualization and program logic. Mainly for this reason scene graph is an ideal datastructure for rendering graphics remotely.

Architecture

org.simantics.scenegraph is designed to be used for rendering graphics locally on desktop application, but also remotely from server to client. For this reason, it has some features that a normal scene graph implementation doesn't have.

In org.simantics.scenegraph, the scene graph datastructure is a tree consisting of ParentNodes and Nodes.

• Node is a leaf in the tree, thus it cannot have children, but it should be used to render visible objects.
• ParentNode is used to group a set of nodes (children) and to give some common parameters for the children.

To enable scene graph datastructure synchronization from a server to a client, a mechanism for detecting changes in the datastructure is required. For this purpose, all changes to Node variables must be made through a public method (usually setter). These methods that change local variables are marked with annotations, thus the solution is very similar to what Hibernate uses with entities.

Node methods can have four different annotations, three for using scene graph remotely, and one for synchronizing node variables automatically from the graph:

• SyncField(..) When a method annotated with this is called, the fields listed in the annotation are updated to the client side (if running in remote mode)
• ServerSide Methods with ServerSide annotation are intended to be executed on server-side, thus it can access resources outside the scenegraph datastructure (using interfaces)
• ClientSide Methods with this annotation are executed on client-side, thus it should only access resources inside the scenegraph datastructure
• PropertySetter This annotation is used to map graph properties to node automatically. This annotation can only be used in setter methods, that take one argument. Notice that the argument type must match with the object that is read from the graph (i.e., for double values you must use Double type instead of double, float, or Float).

Without these annotations, the Node will not work properly when using scenegraph remotely.

These annotations (except PropertySetter) are handled with cglib MethodInterceptor which is a proxyclass for the actual class implementation, and contains an interceptor for performing actions before and after the actual method is called. For this reason, the Node classes should not be instantiated directly, but through ParentNode.addNode method. Cglib support is implemented in org.simantics.scenegraph.remote plugin, and thus org.simantics.scenegraph plugin does not depend on cglib.

ParentNode and Node classes implement only functionalities for creating and modifying the scene graph datastructure but do not contain any methods for rendering.

2D

Java 2D API compatible implementation of the scene graph is in org.simantics.scenegraph.g2d package:

• G2DNode is a baseclass for all leaf nodes that perform 2D rendering
• G2DParentNode is a baseclass for the 2D compatible parent nodes
• IG2DNode is a common interface for G2DNode and G2DParentNode which should be used when traversing through the data structure.
• G2DSceneGraph G2DParentNode implementation containing event listeners and some extra features. This should always be the root component of a G2D scene graph datastructure.

Example G2DNode:

public class BackgroundNode extends G2DNode {
    private static final long serialVersionUID = -1L;	
    protected Rectangle2D bounds = null;
	
    @SyncField("bounds")
    public void setBounds(Rectangle2D bounds) {
        this.bounds = bounds;
    }
	
    @Override
    public void render(Graphics2D g) {
        g.fill(bounds);
    }
}

Because org.simantics.scenegraph is an independent scene graph datastructure, it can be rendered as such. Rendering can be performed by calling G2DSceneGraph.render(Graphics2D g2d) method. No changes are allowed to the scene graph while the structure is rendered (and rendering methods should not make any changes to the structure either, excluding transient node fields that are not synchronized between server and client).

3D

JOGL compatible implementation of the scenegraph can be found under package org.simantics.scenegraph.g3d.jogl in a separate plug-in org.simantics.scenegraph.g3d.

• JOGLNode
• JOGLParentNode
• JOGLPaintable
• JOGLSceneGraph

See org.simantics.scenegraph.g3d.eclipse.JOGLLocalView for an example how to use org.simantics.scenegraph with OpenGL.

Serialization

For serializing the scene graph datastructure there is class called org.simantics.scenegraph.remote.Serializer. The Serializer simply listens for any changes in the scene graph, and translates those changes into events which are then serialized and sent to ObjectOutputStream. To enable Serializer to listen for changes in the scene graph, the scene graph root node must be initialized with cglib. For this purpose there is a static method in Serializer:

 G2DSceneGraph root = Serializer.newRemoteSceneGraphNode(G2DSceneGraph.class);
 Serializer serializer = new Serializer(root, input, output);
 serializer.start();

After this, the root node can be used normally. Any child added to it will be created by cglib automatically, hence Serializer works automatically with these whole scene graph data structure.

On the client side, Serializer can be directly attached to object streams, and it will create G2DSceneGraph object for you:

 Serializer serializer = new Serializer(input, output);
 serializer.start();
 G2DSceneGraph root = (G2DSceneGraph)serializer.getRoot();

Because the system works asynchronously, you should actually attach UpdateListener to serializer in real case.

Notice that you have to attach the serializer to the scene graph before any other operation. Serializer is an instance of PropertyChangeListener which will be set to listen the changes in the scene graph datastructure. If there are changes made before the serializer is attached, these changes will not be serialized.

Serializer uses ObjectInput and ObjectOutput to serialize and unserialize the scene graph, thus it can be used to write the scenegraph to a file and from a file, or to synchronize the scenegraph in a client-server system.

Using scenegraph

• Create scenegraph

G2DSceneGraph sg = new G2DSceneGraph();
sg.setRootPane(instanceOfJComponent); // In case you want to use swing components, you must set rootPane.
// Register event handlers
addKeyListener(sg);
addMouseWheelListener(sg);
addMouseListener(sg);
addMouseMotionListener(sg);
addFocusListener(sg);

• Create node

ShapeNode shape = sg.addNode(ShapeNode.class);
shape.setShape(new Rectangle2D.Double(10, 10, 100, 100));
shape.setColor(Color.RED);

• Render

sg.render(g2d);

• Move shape

shape.setTransform(AffineTransform.getTranslateInstance(5, 5));

• Updating node from stateless object

ShapeNode shape = sg.getOrCreateNode("bounds", ShapeNode.class);
shape.setShape(new Rectangle2D.Double(10, 10, 100, 100));
shape.setColor(Color.RED);

Scenegraph in Simantics

• Each canvas participant that wants to draw something should implement methods annotated with @SGInit and @SGCleanup

public class Participant extends AbstractCanvasParticipant {
   protected ShapeNode node = null;

   @SGInit
   public void initSG(G2DParentNode parent) {
       node = parent.addNode(ShapeNode.class);
       node.setZIndex(PAINT_PRIORITY);
       node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
   }

   @SGCleanup
   public void cleanupSG() {
       node.remove();
       node = null;
   }
}

• Optionally, you can specify a designation for @SGInit that specifies how you want your nodes to be transformed, i.e. in control space or in canvas space. The default value is canvas space. Use the scene graph viewer to understand how this affects your scene graph.

@SGInit(designation = SGDesignation.CONTROL)

• Each diagram element can implement interface SceneGraph to give their own contribution to the scene graph.

public class ElementSGNode implements SceneGraph {
   public static final Key SG_NODE = new SceneGraphNodeKey(Node.class, "SG_NODE");

   @Override
   public void init(final IElement e, G2DParentNode parent) {
       ShapeNode node = (ShapeNode) e.getHint(SG_NODE);
       if (node == null) {
           node = parent.addNode(ShapeNode.class);
           e.setHint(ElementHints.KEY_SG_NODE, node);
       }
       node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
   }

   @Override
   public void cleanup(IElement e) {
       Node node = e.removeHint(SG_NODE);
       if (node != null)
           node.remove();
   }
}

org.simantics.scenegraph features

org.simantics.scenegraph package contains lots of readymade nodes and features. For a list of g2d nodes, see org.simantics.scenegraph.g2d.nodes.

Swing support

It is possible to integrate Swing components to 2D scenegraph by using org.simantics.scenegraph.g2d.nodes.swing.ComponentNode class. See for example MonitorNode or Trend2DNode for an example.

Eclipse view

org.simantics.scenegraph.eclipse package contains local and remote view of 2D and 3D scene graphs. You can try out these demos by executing the project as Eclipse Application and choosing the views "SceneGraph Remote View", "SceneGraph Local View", "JOGL SceneGraph Local View", "JOGL SceneGraph Remote View".

2D-3D integration

It is possible to create a G2DParentNode that works as a wrapper between 2D and 3D scenegraphs. This is not implemented yet.

Node Bounds

In order to perform any kind of rendering optimization in the future some kind of spatial subdivision methods need to be employed. For example Trolltech QT GraphicsView [1] implements BSP-based partitioning to accelerate interaction and rendering in its 2D systems. Bounds information is also useful for picking facilitation.

Currently scene graph nodes do have bounds information, and it is used in rendering to filter off nodes that are not visible in the window. However, this information is not currently used to render view partially. Partial rendering is something that needs to be implemented in the future.

Implementation

• Each node has a method getBoundsInLocal() which returns a Rectangle2D instance describing the 2D boundaries of the node in the element's local coordinate system.
• Node base implementation classes have a method getBounds() that returns the node boundaries with local transforms applied.
• Nodes that have children (i.e., a parent node that does not draw anything) will return union of its childrens bounds.
• G2DSceneGraph root node will cache bounds information.
• Bounds will need caching in order to be efficiently implemented in parent nodes. One option is to implement a mechanism for propagating transformation changes up in the node tree to keep parent nodes aware of transformation changes in their children. Possily implement a listener in parent nodes that can be registered into any transformable node.

Main interfaces

ParentNode

org.simantics.scenegraph.ParentNode

Add child node

Add node simply creates a new node object with the given class and sets it as a child of the parent node.

 public final <TC> TC addNode(Class<TC> a);
 public <TC> TC addNode(String id, Class<TC> a);

Get or create node

If you are not sure if the node already exists, you should give a unique id for the node, and check if the node exists in the scene graph (actually you can only check if its a child of the parent node). For this purpose, there is a method called getOrCreateNode that simply checks if the the node exists and returns it if it does. Otherwise the method will create a new node using addNode.

 public final <TC> TC getOrCreateNode(String id, Class<TC> a);

Remove whole subtree

To remove node and all its children from the scenegraph, a method called remove can be called. This removes node from the parent, and disposes itself and its children. After this method is called, the node and its children are disposed, and thus should not be used anymore.

  public void remove();

Delete node

Delete method can be used to remove a single node from the scene graph tree, so that its children will still be part of the scene graph. This method basically moves nodes children under its parent, and disposes itself. After this method is called, only the node is disposed.

 public void delete();

Remove all children

To remove nodes children, you can use removeNodes-method. This method removes and disposes all children of the node (and their children).

 public final void removeNodes();

INode

org.simantics.scenegraph.INode

Append parent node

To add a node in the middle of the scene graph tree structure, you can use appendParent-method. Basically this method creates a new parent node, adds it as a child of the original parent. Then the node is removed from its parent and added under its new parent.

 public <TC> TC appendParent(String id, Class<TC> nc);

Questions

How to implement Sysdyn -scenario

In this scenario, diagram consists of nodes and arcs between them. Nodes are text boxes that scale according to the text inside them. Arcs go from the center of one node to another so that it clipped to the edges of the nodes. An arrow is drawn in the head of the arc (at the point where the arc is clipped). The following editing operations are supported

1. Changing the name of a node
2. Moving a node. The arcs should be updated during the operation.
3. Changing the curvature of a arc. This is done by dragging some point on the arc to other point.

See [2]. The operations should be reponsive so an implementation that does not need server for updates during the operations is preferred.

Specific questions:

• Can nodes refer to each other? For example, can arc be implemented as

public class ArcNode extends G2DNode {
    protected IConnectable tail = null;
    protected IConnectable head = null;
    protected double angle = 0.0;

    @Override
    public void render(Graphics2D g) {
        calculate an arc from tail to head with given angle
        clip the arc to the clip bounds of the tail and the head
        (or check if connectables have been changed and update the arc if necessary)
    }
}

where

public interface IConnectable {
    double getX();
    double getY();
    Rectangle2D getClipBounds();
}

and

public class TextNode extends G2DNode implements IConnectable { ... }

...answer

In theory, nodes could refer to each other. However, after serialization the references wouldn't work, thus the references must be id based:

  public class ArcNode extends G2DNode {
    protected String tailId = null;
    protected String headId = null;
    protected double angle = 0.0;

    @SyncField("tailId")
    public void setTailId(String id) {
        this.tailId = id;
    }
    @SyncField("headId")
    public void setHeadId(String id) {
        this.headId = id;
    }
    @SyncField("angle")
    public void setAngle(double angle) {
        this.angle = angle;
    }
    @Override
    public void render(Graphics2D g) {
        calculate an arc from tail to head with given angle
        clip the arc to the clip bounds of the tail and the head
        (or check if connectables have been changed and update the arc if necessary)
    }
}

• How derived fields in the nodes are updated when server updates the client-side node?

public class TextNode extends G2DNode implements IConnectable {
    // The properties of the node
    protected double centerX = 0.0;
    protected double centerY = 0.0;
    protected String label = "";

    // Auxiliary values computed in updateDerivedFields
    protected Rectangle2D clipBounds = null;	   
    protected float textX;
    protected float textY;

    private void updateDerivedFields() {
        ...
    }
}

...answer:

The methods in IConnectable should be actually implemented by every G2DNode (Hence, we might combine these interfaces). Notice that you cannot use serialization annotations with private methods.

public class TextNode extends G2DNode implements IConnectable {
    // The properties of the node
    protected double centerX = 0.0;
    protected double centerY = 0.0;
    protected String label = "";

    // Auxiliary values computed in updateDerivedFields
    protected Rectangle2D clipBounds = null;	   
    protected float textX;
    protected float textY;

    @SyncField({"clipBounds, textX, textY"})
    protected void updateDerivedFields() {
        // Update fields, but create clone of clipBounds to enable comparison between old and new value
    }
} 

• How the operation 3 is implemented in client-side? For example, when an arc node notifies that it is being dragged, it should prevent other nodes from getting any mouse events. It should also have some place to store the state of the current operation so that this kind of transient structures do not have to be stored to the nodes. In the applet, this is implemented as:

   class Mover implements ActionHandler {
       public void handleDrag(double x, double y) {
           angle = Arcs.angleOfArc(tail.getX(), tail.getY(), x, y, head.getX(), head.getY());
       }
       public void handleRelease(double x, double y) {}        
   }

   @Override
   public ActionHandler handlePress(double x, double y) {
       double dx = x-cx;
       double dy = y-cy;
       double dist = dx*dx + dy*dy;
       if(dist < (r+SELECTION_TOLERANCE)*(r+SELECTION_TOLERANCE) &&
           dist > (r-SELECTION_TOLERANCE)*(r-SELECTION_TOLERANCE)) {
           double angle = Arcs.normalizeAngle(Math.atan2(-dy, dx));
           if(Arcs.areClockwiseOrdered(angle0, angle, angle1))
               return new Mover();
       }
       return null;
   }

...answer

public class ArcNode extends G2DNode {
   protected double angle;
   protected boolean drag = false;

   public void handleEvent(AWTEvent event) {
       if(!(event instanceof MouseEvent)) return;
       MouseEvent me = (MouseEvent)event;
       if(me.getID() == MouseEvent.MOUSE_PRESSED) {
           double dx = me.getPoint().getX()-cx;
           double dy = me.getPoint().getY()-cy;
           double dist = dx*dx + dy*dy;
           if(dist < (r+SELECTION_TOLERANCE)*(r+SELECTION_TOLERANCE) &&
               dist > (r-SELECTION_TOLERANCE)*(r-SELECTION_TOLERANCE)) {
               double angle = Arcs.normalizeAngle(Math.atan2(-dy, dx));
               if(Arcs.areClockwiseOrdered(angle0, angle, angle1))
                   drag = true;
           }
           me.consume();
       } else if(me.getID() == MouseEvent.MOUSE_DRAGGED && drag == true) {
           G2DNode tail = findNode(tailId); // NOTE: findMode not implemented yet
           G2DNode head = findNode(headId);
           angle = Arcs.angleOfArc(tail.getX(), tail.getY(), me.getPoint().getX(), me.getPoint().getY(), head.getX(), head.getY());
           me.consume();
       } else if(me.getID() == MouseEvent.MOUSE_RELEASED) {
           drag = false;
       }
   }
}

Development Ideas

Node ⇔ ID mapping

(ticket:871)

Just as in DOM implementations the client has the possibility to search for document nodes by their ID, linking between scene graph nodes could be achieved through the use of scene graph -locally unique ID's. Just as in DOM, the id attribute is not mandatory. Lookup will only be possible if an ID is provided.

A lookup facility could again be made available in the scene graph root node. Considering performance, it may become a bottleneck if all ID references would need to first traverse to the scene graph root node to use the ID lookup facility. One way to alleviate this problem is to have a transient root node cache in each node. Obviously this will make the scene graph more space-consuming.

Links

• org.simantics.g2d
• org.simantics.diagram
• cglib
• SVG Salamander

Current Development

• Open Tickets
• Closed Tickets

See also

• TP:Animated Symbols

Contact

juha-pekka.laine@semantum.fi