This is the source archive for smplayer a Linux+OpenGL based LUA scriptable mediaplayer that supports HD video playback and custom digital IO controls; it was used as Digital Retina media art installation by Jill Scott in Version 2.0.

Index with all the ZIP files is here: http://www.ferzkopp.net/Software/smplayer2/

The software is © Andreas Schiffler, aschiffler at ferzkopp.net, 4 Jan 2009 but released under a liberal OSS license.

 

Documentation

 

Introduction

The smplayer2 software was created as a frontend driver for digital signage project and has since been used in some media-art installations.
It is based on a Linux PC with OpenGL graphics acceleration.

Features:
– allows for flat-field calibrated synchronized playback on up to 2 screens
– can handle several SD and HD streams concurrently
– contains many commands for drawing graphics and text to screen
– contains many utility commands for controlling display script (i.e. parallel port IO, http interface)
– many image sources (images, video clips, webcam, VNC)
– based on the LUA scripting language
– runs stable for long times
Limitations (as per Ver 2.0 released 4 Jan 2009):
– video must be converted into proprietary file format (stream of compressed textures) before playback. This is done using a custom mplayer binary and the files may consume a lot of disk space.
– Only partially documented LUA API
– Scripts are not easy to program and debug; lack of IDE.
– Currently no (or very limited) audio support.
– … probably more issues ☺ …

Other than that, it is a very unique and capable piece of software that can help solve unique media display problems in a neat way.

 

General Software Design

The software implements a multithreaded scriptable “game loop”. The graphics thread controls the drawing of OpenGL textures at a fixed rate to the physical screen. The drawing algorithm allows for flatfield calibration (keystone). The program thread executes the media script and performs all “work” in controller- and visualizer-coroutines (not real threads). These make callbacks to scripts wich control visualizers.
[Diagram Here]

LUA Scripting Engine

smplayer2 uses the LUA programming engine for screen and media setup and to control the display while the player is running.
Lua is a very compact, relatively fast and easily extensible functional programming language. Smplayer2 extends it with the following APIs:
– smplayer2 specific objects and functions
– utility APIs (byte buffer, bit manupulation, compression, etc.)
– curl API for web/internet access
– various utility functions written in LUA code

 

Compiling smplayer2

The smplayer2 code is designed to compile and run on a 32bit Linux installation (i.e. last used version was Mandriva Linux 2008.1). Since the player requires hardware accelerated OpenGL support, the proprietary drivers (i.e. from Nvidia) are typically required.
The core mediaplayer depends on many secondary libraries common to a Linux distribution. These should be installed as development packages prior to compiling smplayer2:
– SDL : simple direct media lib, core for graphics
– SDL_gfx : graphics primitives
– SDL_image : image loading
– SDL_net : network primitives
– SDL_ttf : font support and text drawing
– curl : http access, used by lcurl
– expat : XMl parser
– ffmpeg : video codecs, used by custom mplayer
– freetype : TTF rendering, used by SDL_ttf
– jpeg : JPG codec, used by SDL_image
– libbgrab : v4l interface, used by SDL_bgrab
– liblzf : fast compression, used by lbuffer and custom mplayer
– png : PNG codec, used by SDL_image
– openssl : SSL support, used by curl
– zlib : ZIP codec, used by SDL_image
The following locally provided custom libraries need to be also compiled and installed:
– SDL_bgrab : libbgrab interface for SDL, multithreaded v4l-based framegrabber interface
– SDL_vnc : simple multithreaded VNC client for SDL
The LUA scripting engine is provided and extended using these libraries which need to be compiled and installed:
– lua-5.0.2 : custom lua interpreter
– liblzf : fast compression, used by lbuffer
– lbuffer : LUA memory buffers
– lcurl : LUA interface to libcurl
– lthread : LUA threading interface
– lutils : collection of LUA utilities
The video compression format is based on a custom mplayer output mode “glstream”. The source archive contains a custom mplayer version “MPlayer-1.0rc1-glstream” which should be compiled separately.
Data Flow
The following diagram shows the typical data flow and relevant commands between the main API objects:
globals : physical screen calibration
smplayer : logical screen information
ft : frame table, maps of logical frames to the logical screen, corresponds to OpenGL textures
mt : media table, store of in-memory display surfaces
[Diagram Here]

When smplayer is initially started, the frametable and mediatable are initially empty. The script’s Init() callback is executed which generally sets up the physical screen, the calibration and ALL frames and media used during the execution of the program. The the display is opened and the “game loop” entered. Repeated callbacks are made for each frame in the frametable to see if their textures need to be updated. On Mouse or Keyboard events, optional callbacks are executed. If the game loop is terminated the Done() callback is executed.

Script Callback Interface

The smplayer2 executable is started with a LUA script as input. The script needs to provide several callback functions. The following script is the default that needs to be extended by the user:
— Default smplayer2 script
dofile(“utility50.lua”)
function Config()
— Setup display configuration
end
function Init()
— Script initialization
end
function Done()
— Script cleanup
end
function Redraw(frame)
— Redraw of frame ‘frame’ requested
end
function Key(c,s)
— Key c (string), sym s (int) pressed
end
function Mouse(s,x,y,t)
— Mouse clicked (t=1) or
— dragged (t=2) on screen s at coordinate x,y
end

 

Example Program (Code)

— Single frame showing single static image from file
function Init()
— create a frame which will display at 1Hz
myframe=ft.start(1, 1024, 768, 1.00);
— create an empty output pane used to assemble graphics
mypane=mt.create(1024,768);
— load an image into (also creates a pane)
myimg=mt.image(“Media/column.jpg”);
— draw the image into the output pane
mt.blit(mypane, myimg, 0.25*1024,0,0);
— create a text pane
mytext = mt.text(“Hello”, “arial.ttf”,18,0, “white”);
— draw the text into the output pane
mt.blit(mypane, mytext, 10, 10, 1);
— update frame with pane
ft.blit(myframe, mypane, 0, 0, 0);
end
function Redraw(frame)
— update screen with frame
ft.update(myframe);
end
function Done()
— cleanup all objects
myframe=nil
mypane=nil
myimg=nil
mytext=nil
collectgarbage();
end

 

Screen Setup

 

The system supports one or two physical screens.
The application assumes that the X-server has been correctly configured beforehand (i.e. Nvidia driver with OpenGL and TWINVIEW enabled).
Smplayer2 will first parse the Init() section of the input script and then open a window of size [width * screens, height].

Physical to Logical Mapping
Once the physical window has been created, the media script always uses the logical screen coordinates of [0,0 – 1,0] for all frame positioning and drawing commands. This means:
– screen size changes do not affect the playback
– the script must manage all aspect ratios and changes
Debugging of Setup
Additional settings can be specified to assist in the debugging of the application:
smplayer.cursor=1 – enables the mouse cursor
smplayer.outline=1 – enables the display of texture outlines

Flatfield Calibration
To allow for matching the full-screen output of the display device to the actual screen, the system supports a full flatfield calibration (4-point keystone correction). This can be used to:
– pixel match the output of two projectors displaying the same image
– make the images of two adjacent projectors to appear seamless
– fit a screen area to an object
To do that the setCorners command receives an array 8 numbers corresponding to the 4 corners of the screen in the unit coordinate system [0,0 – 1,1].
— default corners
data_screen_one =
{[1]=0,[2]=0,[3]=1,[4]=0,
[5]=1,[6]=1,[7]=0,[8]=1,}
globals.setCorners(1,data_screen_one)
— half size centered corners
data_screen_two =
{[1]=0.25,[2]=0.25,[3]=0.75,[4]=0.25,
[5]=0.75,[6]=0.75,[7]=0.25,[8]=0.75,}
globals.setCorners(2,data_screen_two)

Interactive Screen Setup (Code)
The following Mouse callback function fragment can be used to interactively change the screen calibration on the fly.
— Assumes the variable edit_screen was setup
— and is 1 or 2 if in edit mode or 0 otherwise
— TODO: toggle edit_screen from the keyboard
function Mouse(screen,x,y,t)
— warp screen
if (edit_screen>0) then
if (edit_screen==screen) then
corners = smplayer.getCorners(edit_screen)
if (edit_corner==1) then
corners[1]=x
corners[2]=y
elseif (edit_corner==2) then
corners[3]=x
corners[4]=y
elseif (edit_corner==3) then
corners[5]=x
corners[6]=y
elseif (edit_corner==4) then
corners[7]=x
corners[8]=y
elseif (edit_corner==5) then
— no shift
end
smplayer.setCorners(edit_screen, corners)
— TODO: save the current configuration stored
— in the ‘corners’ variable
end
end
end

Video Compression

 

The smplayer2 video playback uses a stream of compressed textures which are generated by the “glstream” output plugin of a custom mplayer version. The glstream writer supports several GL pixel modes, S3TC compressed textures, alpha channel creation and LZF compression of frames.
The source to the driver can be found in the libvo\vo_glstream.c file.

Mplayer Usage
Example:
mplayer -vo glstream:verbose:dxt5:file=output.dat input.avi
Compression Options:
none (no compression, no output – for benchmarking)
rgba (uncompressed texture, 32bpp)
lum (uncompressed texture, luminance, 8bpp)
luma (uncompressed texture, luminance-alpha, 16bpp)
dxt1 (6bpp)
dxt3 (default, 8bpp)
dxt5 (8bpp)
Other options:
file=filename Save to file <filename> (default: texture.dat)
chromakey Chromakey blue for alpha (default: off)
lzf LZF compress frame (default: off)
verbose Turn on verbose output (default: off)

glstream Fileformat

Fileformat is:
[ file ] = [ fragment 0 ] [ fragment 1 ] . . .
[ fragment X ] = [ header X ] [ frame X ]
where
[ header (22 bytes) ] =
[ magic (4 char) ]
[ flags (uint16) ]
[ width (uint16) ]
[ height (uint16) ]
[ OpenGL format (uint32) ]
[ x uncompressed bytes per frame (uint32) ]
[ y compressed bytes per frame (uint32) ]
[ frame ] =
[ data (x or y bytes per frame depending on flags) ]
[ magic ] =
GLST
[ OpenGL format ] =
GL_RGBA |
GL_LUMINANCE |
GL_LUMINANCE_ALPHA |
GL_COMPRESSED_RGBA_S3TC_DXT1_EXT |
GL_COMPRESSED_RGBA_S3TC_DXT3_EXT |
GL_COMPRESSED_RGBA_S3TC_DXT5_EXT
[ flags ] =
0x0001 – LZF compressed frame
Note: Headers are always repeated before each frame to make reading streams easier.

Sample Conversion Script
The following bash script will convert all input AVI files into “square pixel” SD quality output texture streams cropping the center 480×480 square from the source video frame.
#!/bin/sh
MPLAYER=”/usr/local/bin/mplayer”
PARAMS=”-nosound -noaspect -vop scale=256:256,crop=480:480:80:0 -vo glstream:dxt3:lzf:verbose:file=”
for i in *.avi; do
INPUT=$i
OUTPUT=${i%%.avi}_box.dxt3_lzf
$MPLAYER $PARAMS$OUTPUT $INPUT
done
Note: The target frame size (set with ft.start() in the LUA script) and the texture size (set by the scale option of mplayer) should match for best performance during playback.

Smplayer2 LUA API

Global API
Namespace
globals.xyz
smplayer.xyz
Properties
width (Get, Set)
height (Get, Set)
screens (Get, Set)
cursor (Get, Set)
outline (Get, Set)
draw (Get, Set)
Functions
create() : create object and make it global (automatically called by smplayer2)
list() : displays current smplayer configuration for debugging purposes
float[8] = getCorners(screen) : returns current display positions of screens
setCorners(screen, float[8]) : set display positions of screens for flatfield calibration
quit() : quit game loop and exit smplayer

Frametable API
Namespace
ft.xyz
Properties
screens (get, set) : the screen to place the frame onto
tw (get) : the texture width; power of two
th (get) : the texture height; power of two
r (get, set) : red filter, range=[0.0 – 1.0], default=1.0
g (get, set) : green filter, range=[0.0 – 1.0], default=1.0
b (get, set) : blue filter, range=[0.0 – 1.0], default=1.0
a (get, set) : transparency of frame, range=[0.0 – 1.0], default=1.0
rate (get, set) : update framerate in Hz
x (get, set) : top right X position on logical screen, range=[0.0 – 1.0]
y (get, set) : top right Y position on logical screen, range=[0.0 – 1.0]
w (get, set) : width on logical screen, range=[0.0 – 1.0]
h (get, set) : height on logical screen, range=[0.0 – 1.0]
Functions
start(frame_id, texture_width, texture_height, framerate_hz) : creates new frame object in frametable
list() : lists all frames in frametable for debugging purposes

stop(frame_id) : deletes frame from frametable
clone(source_frame_id, target_frame_id) : sets the content to be cloned from existing frame
unclone(frame_id) : disables cloning for a frame
delay(delay_milliseconds) : do a millisecond delay
color(frame_id, r, g, b, a) : sets the color filters and transparency
rate(frame_id, rate) : sets the frame rate for updates
float[8] = getCorners(frame_id) gets the current corner coordinates in the logical coordinate system [0,0 – 1,1]
setCorners(frame_id, float[8]) : sets the current corner coordinates coordinates in the logical coordinate system [0,0 – 1,1]
warp(frame_id, x1,y1,x2,y2,x3,y3,x4,y4) : sets the current corner coordinates coordinates in the logical coordinate system [0,0 – 1,1]; equivalent to setCorners but different call interface
crop(frame_id, float[8]) : : sets new source texture coordinates; used for cropping or enlarging textures
blit(frame_id, media_id, x, y, blendFlag) : transfer mediatable content into frame
s3t_read(frame_id, fileHandle) : read texture into frame; sync IO
s3tc_tioread(frame_id, tioFileHandle) : read texture into frame; multithreaded async IO
save(frame_id, filename) : save current frame to a file for debugging
update(frame_id) : mark frame for texture update when rate interval expires
vncopen(frame_id, serverHost, serverPort, serverEncoding, serverPassword) : open connection to VNC server
vncupdate(frame_id) : update frame from vnc connection
vncclose(frame_id) : close vnc connection
grabopen(frame_id, device, width, height, channel, mode, frequencyRegion, frequencyIndex) : open framegrabber device

grabupdate(frame_id, deinterlaceFlag) : update frame from framegrabber device
grabclose(frame_id) : close framegrabber connection

Mediatable API
Namespace
mt.xyz
Properties
w (Get) : width of media surface
h (Get) : heigh of media surface
cx (Get, Set) : top right X position of clipping rectangle of media surface
cy (Get, Set) : top right Y position of clipping rectangle of media surface
cw (Get, Set) : width of clipping rectangle of media surface
ch (Get, Set) : height of clipping rectangle of media surface
Functions
image(filename) : create new media surface by load image from file
clone(media_id) : create new media surface from existing media
create(width, height) : create new blank media surface
text(text, fontfile, fontsize, fontstyle, color) : create new media surface from text
rotozoom(media_id, angle, zoomFactor) : create new media surface by rotating and zooming existing media surface
list() : display all media surfaces
clear(media_id, alpha) : clear media surface to particular alpha value
fill(media_id, color) : fill media surface with color
width(media_id, w) : set width of media surface
height(media_id, h) : set height of media surface

blit(source_media_id, target_media_id, x, y, useAlphaFlag) : blit source media surface to target media surface at specified position optionally using alpha blending

clip(media_id, x,y,w,h) : set clipping rectangle for drawing operations on media surface

unclip(media_id) : clear clipping rectangle of media surface

SDL_gfx Mapped Functions

pixel(media_id, x, y, color) : draw pixel
hline(media_id, x1, x2, y, color) : draw horizontal line
vline(media_id, x, y1, y2, color) : draw vertical line
rectangle(media_id, x1,y1,x2,y2, color) : draw rectangle
box(media_id, x1,y1,x2,y2, color) : draw filled rectangle
line(media_id, x1,y1,x2,y2, color) : draw line
aaline(media_id, x1,y1,x2,y2, color) : draw antialiased line
circle(media_id, x,y,r, color) : draw circle
aacircle(media_id, x,y,r, color) : draw antialiased circle
filledcircle(media_id, x,y,r, color) : draw filled circle
ellipse(media_id, x,y,rx,ry, color) : draw ellipse
aaellipse(media_id, x,y,rx,ry, color) : draw antialiased ellipse
filledellipse(media_id, x,y,rx,ry, color) : draw filled ellipse
pie(media_id, x,y,rad,start,end, color) : draw pie
filledpie(media_id, x,y,rad,start,end, color) : draw filled pie
trigon(media_id, x1,y1,x2,y2,x3,y3, color) : draw triangle
aatrigon(media_id, x1,y1,x2,y2,x3,y3, color) : draw antialiased triangle
filledtrigon(media_id, x1,y1,x2,y2,x3,y3, color) : draw filled triangle
polygon() : not implemented
aapolygon() : not implemented
filledpolygon() : not implemented
bezier() : not implemented
string(media_id, x,y, text, color) : draw text in 8×8 fixed width font to media surface

smplayer
Tagged on:                 

Leave a Reply

Your email address will not be published. Required fields are marked *