RayScale is a network distributed 3D graphics renderer based on ray tracing. Written in Node.js, RayScale is designed for scaling out using containers. The objective of RayScale as a project is to demonstrate several points:
- Parallel batch processing to break a slow & complex operation into smaller chunks, to gain performance benefits
- How containers can be scaled extremely easily
- Microservices architecture
- Scaling containers over a cluster such as Kubernetes
- Use of Azure services such as Azure Container Instances to serverlessly provide massive scale
- RESTful APIs
- Use of TypeScript with Node.js and Express
What Is Ray Tracing
In computer graphics, ray tracing is a rendering technique for generating an image by tracing the path of light as pixels in an image plane and simulating the effects of its encounters with virtual objects. The technique is capable of producing a very high degree of visual realism, usually higher than that of typical polygon based scanline rendering methods, but at a greater computational cost.
Ray Tracing Features
It is not the goal of this project to create a completely fully featured ray tracer, as the scope of that is almost limitless. RayScale currently provides:
- Primitive objects: Spheres, planes, cubeboids & cylinders
- Complex polygon meshes in .obj format
- Texture mapping; images (PNG) and checker pattern
- Blinn-Phong illumination model with per object surface properties
- Multiple coloured light sources
- Refractive transparency
- Positionable camera, FOV and image output at any resolution
- Job & scene definition language (YAML)
These are some sample images rendered with RayScale
Core Components (Microservices)
RayScale is comprised of two separate microservices, the Controller and the Tracers. Both are written in Node.js using TypeScript, and all interaction to/from these services is via REST API (using Express).
RayScale is intended to be run with a single Controller and one or more Tracers (either on the same machine as the controller (each on different TCP ports) or elsewhere on the network). Tracers can be started & stopped at any point, more added etc. and the Controller keeps track of tracers via health checks much like a network load balancer.
Basic System Architecture
Acts as control point and main interface with RayScale. It provides the API and Web UI for submitting jobs. It also coordinates the Tracers, keeps tracks on which tracers are online etc. The Controller splits up jobs into tasks and sends them to Tracers, and also reassembles & saves the results as they are sent back
Renders and ray traces tasks given to it via the Controller. Each Tracer registers itself with the Controller on startup. The Tracer carries out scene parsing and also the work of actually computing the ray tracing algorithm of the task it has been given. Once completed, the results are POSTed back to the controller as a binary buffer of image data
The Controller provides a simple web UI, available at
http://<controler-addres>:<port>/ui. The UI allows for:
- Job YAML editing and submission
- Viewing job results, rendered images and other outputs
- Viewing list of tracers online
Objects & Terms
As well as Controller & Tracer RayScale has several named objects and concepts which it’s worth understanding:
- JobInput - A YAML document passed to the Controller to start a Job, at a high level it contains three things; job name, image dimensions and a scene definition. Full details are in the job & scene reference guide
- Job - Internal representation of a Job, managed by the controller, holding its status, an array of tasks, start time etc. When a job is completed most of this information is placed in
- Scene - A description of what is to be rendered, consists of sets of parameters defining the camera position, lights and most importantly all the objects that make up the scene. The objects are described by type, position, size and their appearance. Full details are in the job & scene reference guide
- Tasks - The controller divides a Job into Tasks. You can specify how many tasks you want your job split into, if you don’t the system will simply allocate one task per tracer. A good rule of thumb for quickest rendering is to split the scene into twice as many tasks as you have tracers.
The Controller schedules Tasks by creating a queue and sending them off the queue to idle Tracers. Only a single Task is allocated to a Tracer at a time, once a Task is completed another will be sent to the Tracer off the queue.
The Task contains the above Scene definition, the overall image size, and also the size of the sub-slice the task is to generate. The Tracer will only render the portion of the overall image given to it in the Task
As RayScale uses Node.js and is also containerised there are numerous ways you can run RayScale, here are a few that have been tried tested:
- Locally - without Docker
- Locally - with Docker
- In Kubernetes
- In Azure Container Instances
Scene Definition Language
The way Scenes are defined is in YAML, there’s
Limitations and Design Constraints
These constraints are either by design or known issues
- The system only allows for a single Job to be running at any time
- Failure of any one Task, will result in a failed/incomplete job. Job and Task recovery is considered out of scope, and unlikely to be resolvable.
- Tracers do not check to see if the Controller is active, it is assumed the controller is online at all times. Should the Controller be restarted for any reason, all Tracers will also need to be terminated and restarted.