VNC Viewer for the Roku PhotoBridge HD

Summary

I have written a VNC Viewer for the Roku PhotoBridge following the design described below. For more information visit the VncSix main page.

VncSix is a VNC Viewer for the Roku PhotoBridge HD. Currently it can show the remote display, as well as send mouse movements, clicks, and keystrokes to the server. This is enough to perform most tasks from the PhotoBridge remotely.

Future Features

Several changes and improvements are planned to VncSix, in expected implementation order:

Automatic support of any server resolution. [Now available in version 0.7]
Encrypted password storage and on-screen server selection via customized .roku scripts with embedded server names. [Now available in version 0.5]
Allow server mouse movement and clicking using the remote control. [Now available in version 0.4]
Aspect Ratio Preservation to keep the image from looking stretched. [Now available in version 0.6]
Enable shared connection support so it won't disconnect pre-existing viewer. [Now available in version 0.6]
On-screen server selection for a list if no server specified. [Now available in version 0.9]
On-screen keyboard manipulated by the PhotoBridge remote. [Now available in version 1.0]
Full remote-control setup/configuration: .roku extension and full interactive server/password specification.
Shortcut Generation: On demand, create extra VncSix .roku files for the stored servers. They would then be available from the PhotoBridge GUI using a single click.
Local mouse cursor display for non-cursor-displaying servers like on Mac OS X
On-screen messages for progress and errors.
- Remote server disconnect
- Remote server unavailable
Some form of hardware keyboard support, hopefully, depending on keyboard support by Roku hardware.
Audio support. Transport sound from the server at the same time as the video, and play it back on the PhotoBridge in sync with the video.
32-bit color support.

Automatic Support Of Any Server Resolution

VncSix should automatically adapt to any server resolution. To do so the handleVncComm function must be split into a connection phase function (that includes Connect and Initialise) and a RFB handling phase. The connection phase should be called first in VncSixApp::OnMaterialize. Then the hard-coded 1024x768 there could be replaced by the values from si.framebuffer[Width|Height].

Further changes and debugging are needed. This has been impleemnted but it only works properly for 1024x768. 1920x1080 shows only slightly less than half of left side of the screen. 800x600 is completely garbled.

It appears that the ARGB8888 color format has problems with resolutions wider than 1024 pixels. VncSix will be converted to tuse the RGB565 color space. [Update: this is implemented in version 0.7]

Encrypted Password Storage

VncSix should store server/password pairs in a file for later reuse. The password should be encrypted so that causal snooping won't expose it. vncEncryptAndStorePasswd provides this functionality in the file vncauth.h.

With minor changes, the vncauth code can be changed to encrypt and decrypt passwords and return the encrypted/decrypted strings. Then the strings could be stored in the Roku settings/preferences for VncSix. One master string called "ServerList" would list the addresses (including ":1") separated by spaces. Then, for each address in ServerList there would be one entry listing the encrypted password.

There should be a command-line application storepasswd that takes a server string and a password and writes it into the settings. The application should connect to the server using the store password if only one is in the saved settings. If a server is specified on the command-line the application should connect to that server using the stored password if available in the settings. This would allow a one-time setup of the saved passwords usign the command-line tool, and application shortuts to be setup for the different servers, each of which would connect without asking for a password.

Mouse Movement And Clikcing

VncSix allows the direction buttons on the remote to move the mouse cursor, and the select button to send a left-mouse-click.

Mouse movement accelerates as the direction button is held down longer and longer, to allow for large mouse moves in an acceptable amount of time.

One area of investigation is the concept of local and remote mouse cursors. In the current VncSix code, the only mouse command that can be sent to the server is a button press at a certain location. This means that a local mouse cursor would have to be displayed and moved based on arrow buttons. It would be preferable if the remote mouse cursor could be moved, and a button-press event sent that doesn't specify the location but instead uses the remote location. But this may not be supported on the server side. [Update: using the SendPointerEvent function with a zero button mask (all buttons up) it is possible to move the remote mouse cursor without clicking the mouse. This is now implemented.]

Aspect Ratio Preservation

Many computer displays use a 4:3 aspect ratio (1024x768, 800x600, 1280x1024, etc.). However, the Roku PhotoBridge is usually connected to an HDTV, which has a 16x9 aspect ratio (1920x1080 or 1280x720). In the current version (0.5) VncSix stretches the server display to fill the PhotoBridge display, resulting in distortion. This can be temporarily avoided (on Unix at least) by launching the vncserver with a 16x9 aspect ratio, like so:

vncserver -geometry 1024x576

However, VncSix should be changed to automatically render the server image without distortion, letterboxing on the sides if need be. It should be possible to use the m_scalerOutputRect variable to achieve this.

Audio Support

It would be neat if audio could be transferred from the server and played on the PhotoBridge like video is currently. The VNCAudio project accomplished this some time ago. However, the changes aren't in the VncSix code base and would take some effort to merge. Additionally, the server side code appears to be separate and Unix-specific(?), so other solutions would need to be found for other server OSes.

Original Investigation Notes

I have been examining the source code for several VNC viewer flavors, including:

vncviewer for X11: the regular Unix X11 vnc viewer. Uses numerous X header files, none of which are present on the PhotoBridge. Networking code can be used on PhotoBridge.
fbvnc: a rich viewer for iPaq and Zaurus devices running Linux. Writes to a framebuffer, which is close to the VideoScaler RAM. Networking code will work on PhotoBridge since fbvnc is for Linux platforms too.
palmVNC: a compact viewer for PalmOS. This is instructive since it seems to focus on viewing more than input. However, the Palm networking isn't POSIX, apparently. Also there is lots of code to deal with Palm display issues, none of which is of interest here.

Design Notes

In no particular order:

Will write to VideoScaler RAM for the display. This will allow us to use the full 1920x1080 resolution. The normal graphics buffer is much lower resolution.
Cannot use X headers for compilation, they're not present on the PhotoBridge.
Easiest to work with server that sends 24-bit color data (8 bits each of red, green, and blue) since we can easily write that into the 32-bit video scaler RAM (ARGB -- alpha is the other byte)
Could start with fbvnc code, but disable most overlay logic and input logic.

Specific Changes to fbvnc

Remove fbdraw.c: This is for scaling and rotating, we can just show the whole server image unrotated for now.
In fbgl.c, remove calls to redraw_virt. Make sure that global_framebuffer.v_buf is set to the VideoScaler VRAM buffer and that the width/height are also setup properly.
Determine how to massage the data coming in to gl_putbox to get it ready for the ARGB-style VideoScaler VRAM buffer.
In fbvnc.c, remove overlay.h, icons.h, and keyboard.h, to simplify in line with the read-only nature of this first release.

Future Input Possibilities

Could move the mouse cursor with the navigation buttons on the remote, left-click with select.
Alternatively send arrow keys to VNC server if navigation buttons used? Maybe toggle between arrow keys and mouse?
Typing using USB keyboard if ever supported?

Last modified on 23 Nov 2006 by AO