Main Page
From OptIPortal
OptIPortals
Introduction
What is an OptIPortal
In the broadest sense an OptIPortal is a visualization cluster which can be deployed on a variety of hardware platforms. Functionally, OptIPortals are tiled displays, that is, many displays capable of acting as one or many virtual displays. OptIPortals can be used in a wide variety of visualization approaches; viewing high definition static images, video, or in streaming mode across one or more OptIPortals. OptIPortals include both 2D and 3D environments. At Calit2, our 3D OptIPortals use OpenCOVER (by Covise) as the middleware, although we are currently seeing 3D work coming from the CGLX team as well. Our 2D OptIPortals are running either CGLX or SAGE as the middleware. OptIPortals can also be used for video conferencing and collaboration with other OptIPortals or other compatible devices (laptops, H.323 conferencing devices, etc.). The video conferencing interactions are dependent upon the OptIPortal running SAGE or CGLX, at least on as many tiles as you wish to dedicate to SAGE or CGLX. The video conferencing is achieved by using separate sending and receiving machines and then pushing the output to SAGE or CGLX running on the OptIPortal. Please see the hardware requirement section for HD Video Conferencing hardware requirements. All of these actions can be concurrent or persistent, on the OptIPortal. OptIPortal computers (nodes) comprise a cluster capable of general computations as well as display and collaboration management. Middleware Applications: SAGE, DMX, Chromium, CGLX, OpenCOVER(Covise).
Contact Information
For any questions concerning OptIPortals, please send us a note at optiportal@calit2.net.
Example Deployment
Recent Deployments
OptIPortal Deployments {images and specifications included}
System Requirements
System Software
The OptIPortal is built with Rocks, all software is provided via the Rocks installation - the specific OptIPortal components are obtained via the Viz Roll.
Note: after reviewing this information see the Installation and Configure Section later in this page.
Downloads
Download the Kernel Roll and Viz Roll Here
Note: SAGE is included in the Viz Roll, you will want to add the Hiper Roll to be able choose the tools that best fit your needs - see CGLX - Hiper Roll information in the CGLX Middleware section. Note: as of July 28, 2009 download HiperRoll-5.2.0 via the olderversions link on the CGLX downloads page.
Documentation
Rocks Installation Guide
Rocks Viz Roll Installation Guide
Middleware
This is the software that sits on top of Rocks that allows you to display images and/or video on one or more tiles.
Chromium
Available as part of the Viz Roll - see Rocks Viz Roll Installation Guide
Covise
CGLX
CGLX Home
CGLX is now available as a Rocks Roll - specifically the Hiper Roll. This can be downloaded from CGLX Site. Note: as of July 28, 2009 download HiperRoll-5.2.0 via the olderversions link on the CGLX downloads page.
You can also subscribe to the CGLX Mailing List.
SAGE
Available as part of the Viz Roll - see Rocks Viz Roll Installation Guide.
The full documentation set can be found at the SAGE Site.
Hardware
System Hardware
Since Rocks is the software environment upon which the OptIPortal is based, the hardware requirements for the OptIPortal are essentially those for Rocks. Most of the deployments of OptIPortals have been done on commodity hardware, running Intel or AMD processors. Configurations are possible in which each computer in the cluster can drive one, two or more displays depending on the performance and capabilities of the chose graphics interface. OptIPortals can be optimized for specific functionality in terms of processor speed, network bandwidth, storage capacity and memory availability.
OptIPortals whose primary application include streaming media and interactive collaboration are generally built on bandwidth balanced platforms, in which each display has general access to 50% or more of a Gigabit network interface throughput. Such systems are usually multi-core but need not consist of the fastest processors available. Each core generally has a Gigabit of memory available. The goal here is for a balanced bisection between communications speed and processor bandwidth. Computers in these systems computers generally drive 1-2 displays each.
OptIPortals that are maximized for pixel space (large display walls in excess of 30 displays or hundreds of megapixels) are usually driven by fewer computers (that is, each computer may drive 4 or more displays). As such, the computers are generally based on higher speed multi-core processors with 2 Gigabyte or more of memory each and ideally 10GigE network interfaces.
Note: if you will be connecting to the Optiputer network - be sure to use Intel Pro/1000 NICs - these gernally support Jumbo Packets (MTU 9000). The broadcom nics that come with most Dells do not support Jumbo Packets.
Network Hardware
A 1Gb switch is the minimum requirement for inter-nodal communication. We have used a variety of commodity switches for the cluster communication, including Cisco, Force10, SMC and NetGear.
OptIPuter Network
If you wish to connect to the OptIPuter network, be certain to verify network connectivity with your Network Engineer. You will need additional ports on your inter-nodal switch to accomodate the OptIPuter connectivity - depending upon which switch you have purchased, you may choose to use a second switch for this purpose. Please direct any questions to mailto:optiportal@calit2.net.
As for NICs, use Intel Pro/1000 whenever possible - this will make certain that you can use Jumbo Packets (MTU 9000) if you will be connecting t o the Optiputer network.
Graphical Hardware
Displays
Most displays suitable for computing environments will operate, however, if you will be displaying high definition imaging, it is best to use a relatively modern LCD display for your displays. We have typically used 24", 30", 46" and 52" LCDs.
Graphics Cards
Be sure to choose a graphics card that can drive your displays at the desired resolution. You will have to balance heat and power requirements for the systems you have selected (and for the location where the OptIPortal will reside). Proper heat balancing is essential to the longevity of the systems driving the displays. Recent deployments have used nVidia projects, examples include: Quadro NVS 285 series (low), GeForce 8800 (mid) and Quadro FX 5600 (high). Choose the model that best meets your applicatoin needs (and that doesn't exceed your hardware specifications).
Framing
The vendor we have used mosted often for OptIPortal framing is a company named 80/20. Below are two spreadsheets that can be used to generate a parts list for two example OptIPortals, one based on 24" Dell LCDs and the other on 30" Dell LCDs. We have also built frames for 46" and 52" displays - send queries to optiportal@calit2.net
Note: be certain to carefully read the ordering instructions on the spreadsheet - you will be ordering by number of stems (columns) required for your configuration.
How to Assemble the Frame
24-inch Parts List Spreadsheet
30-inch Parts List Spreadsheet
Framing Supply Contacts
- Mike Fanolla
- F & L Industrial Solutions, Inc.
- (858) 271-6732
- fandlindsol@sbcglobal.net
HD Video Conferencing
HD Video Conferencing is achieved by setting up separate sending and receiving machines and then pushing the output to the OptIPortals using one of the two following methods (for both methods our current recommendation is to use a Qvidiuim setup to do the HD streaming to another Qvideum setup at a remote location, see the "HD Video Conferencing Hardware" for specifics).
- CGLX - BlackMagic Card: to use this option a PC with a BlackMagic card is placed on the private internodal network (physically connected to the internodal switch) - the video out signal from the QVideum box is connected to the BlackMagic card - the stream is then sent to the OptIPortal and can be displayed via the StreamWindow in the MediaViewer app in CGLX.
- SAGE API: to use this option the OptIPortal must be running SAGE to receive the output from the HD Video Conferencing machine. The output is then pushed to the OptIPortal using a SAGE API.
HD Video Conferencing Hardware
Commercial VTC Option
A commercial HD VTC solution, such as LifeSize can be used. Here at Calit2, UCSD, we have used the LifeSize Room 200 series - LifeSize example. We then connect the video output to the streaming box (described in the prior section) and direct the output to the wall. The LifeSize Room 200 is just one example of other similar commercial products. The advantages are simplicity and compatibility with other VTC hardware and complete audio support.
Research Option
This option provides research grade video delivery, with flexibility to modify the components according to your needs.
- Video:
- Camera: Panasonic DVCproHD
- Display: Sharp Model: Aquos LC-45GX8U
- Misc: DVI to HDMI cable (connect PC nVidia GeForce 7900 GTX DVI ouput to Sharp break-up box. Only way to obtain HD from Sharp TV)
- Streaming server/receiver:
- Convert firewire packets into ethernet packets: 1 x Qvidium box
- PC to receive, decompress and render audio and video stream:
- Intel quad core CPU @ 2.66 GHz (to avoid bottleneck in the decoding) or better.
- nVidia GeForce 7900 GTX graphics card (to avoid bottleneck in the decoding and rendering) or better.
- 1 GB of RAM or more.
- If you want to record the input stream: a fast SATA2 10000 rpm hard disk
- Intel PRO/1000 MT Dual Port Server network card (NIC) (motherboard NIC is usually not very good quality and can cause packets drop).
- Good audio board (embedded motherboard sound chip is OK).
- Audio Input:
- Noise cancelation microphone, ClearOne Model: ACCUMIC PC
- Mono un-balance to balance : Whirlwind Model: pcDI Direct Box
- Split mono balance to two-channel balance: Pro-Co Sound Model: MS-3
- Misc: 3 x XLR to XLR mono patch cables (will work out lengths with Javier)
- Audio Output:
- Speakers: 2 x Powered Monitor Speaker Yamaha, Model: HS 50M
- Audio mixer & convert stereo RCA to 1/4" phone plug: PEAVEY ELECTRONICS CORPORATION Model: PV6
- Misc:
- Connect mixer to speakers: 2 x 1/4" to 1/4" mono patch cable (length depends on the setting).
- Connect audio output to mixer input: 1 x 3.5mm stereo plug to RCA plug x 2 Y-Cable.
- Lighting:
- A pair of Videossence light fixtures (stand mount)
Installation and Configuration
How do I build a tiled wall or OptIPortal
There is tremendous flexibility in the hardware to be used. See Recent Deployments for a recent list of installations.
The OptIPortal is built with Rocks, all software is provided via the Rocks installation - the specific OptIPortal components are obtained via the Viz Roll.
Download the Software
Download the Kernel Roll and Viz Roll Here
SAGE is included in the Viz Roll - be sure to add the Hiper Roll from http://www.cglx.org - you will then be able to choose the middleware that best suits your needs (review the middleware choices earlier on this page). Note: as of July 28, 2009 download HiperRoll-5.2.0 via the olderversions link on the CGLX downloads page.
Download the Hiper Roll Here
Note: download the kernel and viz roll iso's, boot from this kernel roll CD on your head-node, type front-end at the prompt, you will soon see a screen asking you to Select Your Rolls - select Download from the Network-based Rolls. Be sure to select the following rolls: kernel, base, hpc, os, viz web-server. If you have downloaded the Hiper Roll, be sure to select CD/DVD from the Local Rolls section of the Select Your Rolls screen before proceeding.
If you cannot download network based installation rolls, then when you burn media be sure to add the individual OS roll media in addition to the jumbo DVD. Installing from burned media is fine, but the OS roll information, specifically, will be more complete when doing the network based roll.
Be sure to review the Installation Documentation.
Rocks Installation Guide
Rocks Viz Roll Installation Guide
Installation FAQ
Disable auto-reinstall of tile nodes
tentakel /etc/rc.d/init.d/rocks-grub stop
tentakel /sbin/chkconfig --del rocks-grub
Add broader multicast route for CGLX
For CGLX to function properly create the following to each node, including the head-node - assuming eth0 is the interface for your 10-net:
/etc/sysconfig/network-scripts/route-eth0
The contents should be:
NETMASK0=240.0.0.0
ADDRESS0=224.0.0.0
Restart the network on each node and head-node.
root ssh keys not passed to node at installation
Periodically you may find that you are prompted for a password as root when going from head-node to tile-node.
- Short-term fix:
- copy /root/.ssh/id_rsa and /root.ssh/id_rsa.pub
- be sure to follow the procedure below for installation of new nodes
- Long-term fix:
- you need to make '/root/.ssh/id_rsa.pub' world readable, then
- reinstall your compute nodes.
- # chmod a+r /root/.ssh/id_rsa.pub
Head-node - failed X server
Under 5.2 with the Viz roll, the head-node will often have an X Server failure at first startup after installation - simply remove /etc/X11/xorg.conf and then regenerate - use nvidia-xconfig or example and then restart X (init 3; init 5).
Update nVidia Driver
Download the latest nVidia Driver from http://www.nvidia.com. Place in /share/apps and make the installation script executable. For example:
head-node:
init 3
/share/apps/NVIDIA-Linux-x86_64-185.18.14-pkg2.run --no-network -s > /dev/null
init 5
tile-nodes:
tentakel init 3
tentakel /share/apps/NVIDIA-Linux-x86_64-185.18.14-pkg2.run --no-network -s > /dev/null
tentakel init 5
xorg.conf for CGLX
- Make certain that all nodes have the following in their /etc/X11/xorg.conf (including the head-node):
- Section "Module"
- Load "vnc"
- Load "dbe"
- Load "extmod"
- Load "type1"
- Load "freetype"
- Load "glx"
- EndSection
Tile Layout: Rocks Method - from Viz Roll Docs
The Viz Roll supports the construction of walls with either a single LCD display per machine or multiple displays per machine. Multiple displays can be grouped together as a single X11 display using TwinView with or without Xinerama, or each display can be an independent X11 display. The choice of whether to group a host's physical displays together or not is controlled by the rocks sync tile command.
The default configuration for Rocks is a single physical display per machine. If this is your configuration skip to the next section. If your wall has multiple displays per host you will need to create an XML an XML file that describes the physical layout of your wall. The syntax is very simple and the XML itself maps to the dimensions of the wall. Each col tag describes a vertical column of the wall, with the nodes listed top to bottom. The very first col section lists the Tile nodes comprising the far left side of the wall, and from there the next section moves to the right.
Warning
When connecting a single video card to two display you must connect the primary video connector to the LCD above (or to the left) of the secondary connector. This is a requirement of the SAGE software.
1. This simplest method to create you layout XML is to start from scratch rather than modifying the defaults as is documented in the non-twinview section above.
<wall>
<col>
<display host="tile-0-0:0.0"/>
<display host="tile-0-1:0.0"/>
<display host="tile-0-2:0.0"/>
</col>
<col>
<display host="tile-0-0:0.1"/>
<display host="tile-0-1:0.1"/>
<display host="tile-0-2:0.1"/>
</col>
<col>
<display host="tile-1-0:0.0"/>
<display host="tile-1-1:0.0"/>
<display host="tile-1-2:0.0"/>
</col>
<col>
<display host="tile-1-0:0.1"/>
<display host="tile-1-1:0.1"/>
<display host="tile-1-2:0.1"/>
</col>
<col>
<display host="tile-2-0:0.0"/>
<display host="tile-2-1:0.0"/>
<display host="tile-2-2:0.0"/>
</col>
</wall>
2. After you update your tilelayout XML, now apply it to the database:
# rocks add tile layout layout.xml
3. Finally, reconfigure your tiles:
# rocks sync tile mode=meta
Tile Layout: Creating Customized /etc/X11/xorg.conf
In some cases you may want to customize the xorg.conf on the nodes - many opencover and cglx users prefer this method - but you will have to keep track of your modifications. The easiest method is to run nvidia-settings from the nodes and display the gui on the head-node:
open a root shell on the head-node - ssh to tile-node and run nvidia-settings on the tile-nodes, for example:
head-node# ssh tile-0-0
root@tile-0-0# nvidia-settings --ctrl-display=tile-0-0:0
{save your changes and restart X}
SAGE Startup Issue
- First time startup of SAGE gives: "UnboundLocalError: local variable 'tile' referenced before assignment"
- you must first run the tile setup first: see steps above:
- File_Server startup problems:
- stop and start via advanced window in SAGE startup ui - click on the red X to toggle to green check
- try starting from command line /opt/rocks/bin/python2.4/file_server.py
How to change Public IP on head-node
This is not a complete process, but will work reasonably well for general cluster behavior.
Use the following:
* rocks set host interface ip cluster.domain eth1 <new-IP>
* rocks sync config
If you see python errors from rocks sync config - manually change /etc/sysconfig/ifcfg-eth1 and restart networking - /etc/init.d/network restart - wait a couple of minutes and rerun rocks sync config.
Note: this process is incomplete, but may suffice for most cluster operation, you will still see the old IP on node Kickstarts - to change this try:
* rocks set var Kickstart PublicAddress <new-IP>, this invocation looks correct, but was failing under Rocks 5.2.
The absolute robust way is to prepare a restore roll and re-build using the new IP.
What if you don't have a Public IP for the head-node
We call this road-show mode. Of course, you will need all of your installation media - Rocks Jumbo DVD, etc.
You can just assign a dummy IP address for the Public IP at installation time. After you have PXE booted all your nodes, ssh to each node (there will a delay {dns induced}) and change the IP of the head-node Public to the head-node's 10-net address. The nodes will now communicate properly with the head-node and all will be well. A kluge to be certain, but functional.
SAGE libGLEW Version Mismatch
- When using SAGE after having installed CGLX - there will be libGLEW mismatch - SAGE is looking for libGLEW 1.4, CGLX uses 1.5. Correct by creating 1.4 link to 1.5 lib:
- [head-node] ln -s /usr/local/lib64/libGLEW.so /usr/local/lib64/libGLEW.so.1.4
- [tile-nodes] tentakel ln -s /usr/local/lib64/libGLEW.so /usr/local/lib64/libGLEW.so.1.4
