JME3 Canvas in a Swing GUI

3D games are typically played full-screen, or in a window that takes over the mouse and all inputs. However it is also possible to embed a jME 3 canvas in a standard Swing application.
This can be useful when you create some sort of interactive 3D viewer with a user interface that is more complex than just a HUD: For instance an interactive scientific demo, a level editor, or a game character designer.

  • Advantages:

    • You can use Swing components (frame, panels, menus, controls) next to your jME3 game.

    • The NetBeans GUI builder is compatible with the jMonkeyEngine; you can use it it to lay out the Swing GUI frame, and then add() the jME canvas into it. Install the GUI builder via Tools → Plugins → Available Plugins.

  • Disadvantages:

    • You cannot use SimpleApplication’s default mouse capturing for camera navigation, but have to come up with a custom solution.

Here is the full TestCanvas.java code sample.

Extending SimpleApplication

You start out just the same as for any jME3 game: The base application, here SwingCanvasTest, extends com.jme3.app.SimpleApplication. As usual, you use simpleInitApp() to initialize the scene, and simpleUpdate() as event loop.
The camera’s default behaviour in SimpleApplication is to capture the mouse, which doesn’t make sense in a Swing window. You have to deactivate and replace this behaviour by flyCam.setDragToRotate(true); when you initialize the application:

public void simpleInitApp() {
  // activate windowed input behaviour
  flyCam.setDragToRotate(true);
  // Set up inputs and load your scene as usual
  ...
}

In short: The first thing that is different is the main() method. We don’t call start() on the SwingCanvasTest object as usual. Instead we create a Runnable() that creates and opens a standard Swing jFrame. In the runnable, we also create our SwingCanvasTest game with special settings, create a Canvas for it, and add that to the jFrame. Then we call startCanvas().

Main() and Runnable()

The Swing isn’t thread-safe and doesn’t allow us to keep the jME3 canvas up-to-date. This is why we create a runnable for the jME canvas and queue it in the AWT event thread, so it can be invoked “later” in the loop, when Swing is ready with updating its own stuff.
In the SwingCanvasTest’s main() method, create a queued runnable(). It will contain the jME canvas and the Swing frame.

  public static void main(String[] args) {
    java.awt.EventQueue.invokeLater(new Runnable() {
      public void run() {
         // ... see below ...
      }
    });
  }

Note that you have to use app.enqueue() when modifying objects in the scene from the AWT Event Queue like you have to use java.awt.EventQueue.invokeLater() from other threads (e.g. the update loop) when changing swing elements. This can get hairy quickly if you don’t have a proper threading model planned so you might want to use NiftyGUI as it is embedded in the update loop thread and is also cross-platform compatible (e.g. android etc.).

Creating the Canvas

Here in the run() method, we start the jME application, create its canvas, create a Swing frame, and add everything together.
Specify the com.jme3.system.AppSettings so jME knows the size of the Swing panel that we put it into. The application will not ask the user for display settings, you have to specify them in advance.

AppSettings settings = new AppSettings(true);
settings.setWidth(640);
settings.setHeight(480);

We create our canvas application SwingCanvasTest, and give it the settings. We manually create a canvas for this game and configure the com.jme3.system.JmeCanvasContext. The method setSystemListener() makes sure that the listener receives events relating to context creation, update, and destroy.

SwingCanvasTest canvasApplication = new SwingCanvasTest();
canvasApplication.setSettings(settings);
canvasApplication.createCanvas(); // create canvas!
JmeCanvasContext ctx = (JmeCanvasContext) canvasApplication.getContext();
ctx.setSystemListener(canvasApplication);
Dimension dim = new Dimension(640, 480);
ctx.getCanvas().setPreferredSize(dim);

Note that we have not called start() on the application, as we would usually do in the main() method. We will call startCanvas() later instead.

Creating the Swing Frame

Inside the run() method, you create the Swing window as you would usually do. Create an empty jFrame and add() components to it, or create a custom jFrame object in another class file (for example, by using the NetBeans GUI builder) and create an instance of it here. Which ever you do, let’s call the jFrame window.

JFrame window = new JFrame("Swing Application");
window.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

We create a standard JPanel inside the JFrame. Give it any Layout you wish – here we use a simple Flow Layout. Where the code sample says “Some Swing Component”, this is where you add your buttons and controls.
The important step is to add() the canvas component into the panel, like all the other Swing components.

JPanel panel = new JPanel(new FlowLayout()); // a panel
// add all your Swing components ...
panel.add(new JButton("Some Swing Component"));
...
// add the JME canvas
panel.add(ctx.getCanvas());

OK, the jFrame and the panel are ready. We add the panel into the jFrame, and pack everything together. Set the window’s visibility to true make it appear.

window.add(panel);
window.pack();
window.setVisible(true);

Remember that we haven’t called start() on the jME application yet? For the canvas, there is a special startCanvas() method that you must call now:

canvasApplication.startCanvas();

Clean, build, and run!

Remember, to navigate in the scene, click and drag (!) the mouse, or press the WASD keys. Depending on your game you may even want to define custom inputs to handle navigation in this untypical environment.