{"id":75,"date":"2016-01-02T17:35:18","date_gmt":"2016-01-02T17:35:18","guid":{"rendered":"http:\/\/www.ferzkopp.org\/wordpress\/?p=75"},"modified":"2016-01-08T07:44:29","modified_gmt":"2016-01-08T15:44:29","slug":"smplayer-scriptable-mediaplayer-v2-0","status":"publish","type":"post","link":"https:\/\/www.ferzkopp.net\/wordpress\/2016\/01\/02\/smplayer-scriptable-mediaplayer-v2-0\/","title":{"rendered":"smplayer"},"content":{"rendered":"

This is the source archive for smplayer<\/em> a Linux+OpenGL based LUA scriptable mediaplayer that supports HD video playback and custom digital IO controls; it was used as Digital Retina<\/em> media art installation by Jill Scott in Version 2.0.<\/p>\n

<\/p>\n

Index with all the ZIP files is here: http:\/\/www.ferzkopp.net\/Software\/smplayer2\/<\/a><\/p>\n

The software is \u00a9 Andreas Schiffler, aschiffler at ferzkopp.net, 4 Jan 2009 but released under a liberal OSS license.<\/p>\n

 <\/p>\n

Documentation<\/h4>\n

 <\/p>\n

Introduction<\/h6>\n

The smplayer2 software was created as a frontend driver for digital signage project and has since been used in some media-art installations.
\nIt is based on a Linux PC with OpenGL graphics acceleration.<\/p>\n

Features:
\n– allows for flat-field calibrated synchronized playback on up to 2 screens
\n– can handle several SD and HD streams concurrently
\n– contains many commands for drawing graphics and text to screen
\n– contains many utility commands for controlling display script (i.e. parallel port IO, http interface)
\n– many image sources (images, video clips, webcam, VNC)
\n– based on the LUA scripting language
\n– runs stable for long times
\nLimitations (as per Ver 2.0 released 4 Jan 2009):
\n– video must be converted into proprietary file format (stream of compressed textures) before playback. This is done using a custom mplayer binary and the files may consume a lot of disk space.
\n– Only partially documented LUA API
\n– Scripts are not easy to program and debug; lack of IDE.
\n– Currently no (or very limited) audio support.
\n– \u2026 probably more issues \u263a \u2026<\/p>\n

Other than that, it is a very unique and capable piece of software that can help solve unique media display problems in a neat way.<\/p>\n

 <\/p>\n

General Software Design<\/h6>\n

The software implements a multithreaded scriptable \u201cgame loop\u201d. The graphics thread controls the drawing of OpenGL textures at a fixed rate to the physical screen. The drawing algorithm allows for flatfield calibration (keystone). The program thread executes the media script and performs all \u201cwork\u201d in controller- and visualizer-coroutines (not real threads). These make callbacks to scripts wich control visualizers.
\n[Diagram Here]<\/p>\n

LUA Scripting Engine<\/h6>\n

smplayer2 uses the LUA programming engine for screen and media setup and to control the display while the player is running.
\nLua is a very compact, relatively fast and easily extensible functional programming language. Smplayer2 extends it with the following APIs:
\n– smplayer2 specific objects and functions
\n– utility APIs (byte buffer, bit manupulation, compression, etc.)
\n– curl API for web\/internet access
\n– various utility functions written in LUA code<\/p>\n

 <\/p>\n

Compiling smplayer2<\/h6>\n

The smplayer2 code is designed to compile and run on a 32bit Linux installation (i.e. last used version was Mandriva Linux 2008.1). Since the player requires hardware accelerated OpenGL support, the proprietary drivers (i.e. from Nvidia) are typically required.
\nThe core mediaplayer depends on many secondary libraries common to a Linux distribution. These should be installed as development packages prior to compiling smplayer2:
\n– SDL : simple direct media lib, core for graphics
\n– SDL_gfx : graphics primitives
\n– SDL_image : image loading
\n– SDL_net : network primitives
\n– SDL_ttf : font support and text drawing
\n– curl : http access, used by lcurl
\n– expat : XMl parser
\n– ffmpeg : video codecs, used by custom mplayer
\n– freetype : TTF rendering, used by SDL_ttf
\n– jpeg : JPG codec, used by SDL_image
\n– libbgrab : v4l interface, used by SDL_bgrab
\n– liblzf : fast compression, used by lbuffer and custom mplayer
\n– png : PNG codec, used by SDL_image
\n– openssl : SSL support, used by curl
\n– zlib : ZIP codec, used by SDL_image
\nThe following locally provided custom libraries need to be also compiled and installed:
\n– SDL_bgrab : libbgrab interface for SDL, multithreaded v4l-based framegrabber interface
\n– SDL_vnc : simple multithreaded VNC client for SDL
\nThe LUA scripting engine is provided and extended using these libraries which need to be compiled and installed:
\n– lua-5.0.2 : custom lua interpreter
\n– liblzf : fast compression, used by lbuffer
\n– lbuffer : LUA memory buffers
\n– lcurl : LUA interface to libcurl
\n– lthread : LUA threading interface
\n– lutils : collection of LUA utilities
\nThe video compression format is based on a custom mplayer output mode \u201cglstream\u201d. The source archive contains a custom mplayer version \u201cMPlayer-1.0rc1-glstream\u201d which should be compiled separately.
\nData Flow
\nThe following diagram shows the typical data flow and relevant commands between the main API objects:
\nglobals : physical screen calibration
\nsmplayer : logical screen information
\nft : frame table, maps of logical frames to the logical screen, corresponds to OpenGL textures
\nmt : media table, store of in-memory display surfaces
\n[Diagram Here]<\/p>\n

When smplayer is initially started, the frametable and mediatable are initially empty. The script\u2019s Init() callback is executed which generally sets up the physical screen, the calibration and ALL frames and media used during the execution of the program. The the display is opened and the \u201cgame loop\u201d entered. Repeated callbacks are made for each frame in the frametable to see if their textures need to be updated. On Mouse or Keyboard events, optional callbacks are executed. If the game loop is terminated the Done() callback is executed.<\/p>\n

Script Callback Interface<\/h6>\n

The smplayer2 executable is started with a LUA script as input. The script needs to provide several callback functions. The following script is the default that needs to be extended by the user:
\n— Default smplayer2 script
\ndofile(“utility50.lua”)
\nfunction Config()
\n— Setup display configuration
\nend
\nfunction Init()
\n— Script initialization
\nend
\nfunction Done()
\n— Script cleanup
\nend
\nfunction Redraw(frame)
\n— Redraw of frame ‘frame’ requested
\nend
\nfunction Key(c,s)
\n— Key c (string), sym s (int) pressed
\nend
\nfunction Mouse(s,x,y,t)
\n— Mouse clicked (t=1) or
\n— dragged (t=2) on screen s at coordinate x,y
\nend<\/p>\n

 <\/p>\n

Example Program (Code)<\/h6>\n

— Single frame showing single static image from file
\nfunction Init()
\n— create a frame which will display at 1Hz
\nmyframe=ft.start(1, 1024, 768, 1.00);
\n— create an empty output pane used to assemble graphics
\nmypane=mt.create(1024,768);
\n— load an image into (also creates a pane)
\nmyimg=mt.image(“Media\/column.jpg”);
\n— draw the image into the output pane
\nmt.blit(mypane, myimg, 0.25*1024,0,0);
\n— create a text pane
\nmytext = mt.text(\u201cHello\u201d, \u201carial.ttf\u201d,18,0, “white”);
\n— draw the text into the output pane
\nmt.blit(mypane, mytext, 10, 10, 1);
\n— update frame with pane
\nft.blit(myframe, mypane, 0, 0, 0);
\nend
\nfunction Redraw(frame)
\n— update screen with frame
\nft.update(myframe);
\nend
\nfunction Done()
\n— cleanup all objects
\nmyframe=nil
\nmypane=nil
\nmyimg=nil
\nmytext=nil
\ncollectgarbage();
\nend<\/p>\n

 <\/p>\n

Screen Setup<\/h6>\n

 <\/p>\n

The system supports one or two physical screens.
\nThe application assumes that the X-server has been correctly configured beforehand (i.e. Nvidia driver with OpenGL and TWINVIEW enabled).
\nSmplayer2 will first parse the Init() section of the input script and then open a window of size [width * screens, height].<\/p>\n

Physical to Logical Mapping<\/strong>
\nOnce the physical window has been created, the media script always uses the logical screen coordinates of [0,0 \u2013 1,0] for all frame positioning and drawing commands. This means:
\n– screen size changes do not affect the playback
\n– the script must manage all aspect ratios and changes
\nDebugging of Setup<\/strong>
\nAdditional settings can be specified to assist in the debugging of the application:
\nsmplayer.cursor=1 \u2013 enables the mouse cursor
\nsmplayer.outline=1 \u2013 enables the display of texture outlines<\/p>\n

Flatfield Calibration<\/strong>
\nTo allow for matching the full-screen output of the display device to the actual screen, the system supports a full flatfield calibration (4-point keystone correction). This can be used to:
\n– pixel match the output of two projectors displaying the same image
\n– make the images of two adjacent projectors to appear seamless
\n– fit a screen area to an object
\nTo do that the setCorners command receives an array 8 numbers corresponding to the 4 corners of the screen in the unit coordinate system [0,0 \u2013 1,1].
\n— default corners
\ndata_screen_one =
\n{[1]=0,[2]=0,[3]=1,[4]=0,
\n[5]=1,[6]=1,[7]=0,[8]=1,}
\nglobals.setCorners(1,data_screen_one)
\n— half size centered corners
\ndata_screen_two =
\n{[1]=0.25,[2]=0.25,[3]=0.75,[4]=0.25,
\n[5]=0.75,[6]=0.75,[7]=0.25,[8]=0.75,}
\nglobals.setCorners(2,data_screen_two)<\/p>\n

Interactive Screen Setup (Code)<\/strong>
\nThe following Mouse callback function fragment can be used to interactively change the screen calibration on the fly.
\n— Assumes the variable edit_screen was setup
\n— and is 1 or 2 if in edit mode or 0 otherwise
\n— TODO: toggle edit_screen from the keyboard
\nfunction Mouse(screen,x,y,t)
\n— warp screen
\nif (edit_screen>0) then
\nif (edit_screen==screen) then
\ncorners = smplayer.getCorners(edit_screen)
\nif (edit_corner==1) then
\ncorners[1]=x
\ncorners[2]=y
\nelseif (edit_corner==2) then
\ncorners[3]=x
\ncorners[4]=y
\nelseif (edit_corner==3) then
\ncorners[5]=x
\ncorners[6]=y
\nelseif (edit_corner==4) then
\ncorners[7]=x
\ncorners[8]=y
\nelseif (edit_corner==5) then
\n— no shift
\nend
\nsmplayer.setCorners(edit_screen, corners)
\n— TODO: save the current configuration stored
\n— in the \u2018corners\u2019 variable
\nend
\nend
\nend<\/p>\n

Video Compression<\/h6>\n

 <\/p>\n

The smplayer2 video playback uses a stream of compressed textures which are generated by the \u201cglstream\u201d output plugin of a custom mplayer version. The glstream writer supports several GL pixel modes, S3TC compressed textures, alpha channel creation and LZF compression of frames.
\nThe source to the driver can be found in the libvo\\vo_glstream.c file.<\/p>\n

Mplayer Usage<\/strong>
\nExample:
\nmplayer -vo glstream:verbose:dxt5:file=output.dat input.avi
\nCompression Options:
\nnone (no compression, no output – for benchmarking)
\nrgba (uncompressed texture, 32bpp)
\nlum (uncompressed texture, luminance, 8bpp)
\nluma (uncompressed texture, luminance-alpha, 16bpp)
\ndxt1 (6bpp)
\ndxt3 (default, 8bpp)
\ndxt5 (8bpp)
\nOther options:
\nfile=filename Save to file <filename> (default: texture.dat)
\nchromakey Chromakey blue for alpha (default: off)
\nlzf LZF compress frame (default: off)
\nverbose Turn on verbose output (default: off)
\n
\nglstream Fileformat<\/strong>
\nFileformat is:
\n[ file ] = [ fragment 0 ] [ fragment 1 ] . . .
\n[ fragment X ] = [ header X ] [ frame X ]
\nwhere
\n[ header (22 bytes) ] =
\n[ magic (4 char) ]
\n[ flags (uint16) ]
\n[ width (uint16) ]
\n[ height (uint16) ]
\n[ OpenGL format (uint32) ]
\n[ x uncompressed bytes per frame (uint32) ]
\n[ y compressed bytes per frame (uint32) ]
\n[ frame ] =
\n[ data (x or y bytes per frame depending on flags) ]
\n[ magic ] =
\nGLST
\n[ OpenGL format ] =
\nGL_RGBA |
\nGL_LUMINANCE |
\nGL_LUMINANCE_ALPHA |
\nGL_COMPRESSED_RGBA_S3TC_DXT1_EXT |
\nGL_COMPRESSED_RGBA_S3TC_DXT3_EXT |
\nGL_COMPRESSED_RGBA_S3TC_DXT5_EXT
\n[ flags ] =
\n0x0001 – LZF compressed frame
\nNote: Headers are always repeated before each frame to make reading streams easier.<\/p>\n

Sample Conversion Script<\/strong>
\nThe following bash script will convert all input AVI files into \u201csquare pixel\u201d SD quality output texture streams cropping the center 480×480 square from the source video frame.
\n#!\/bin\/sh
\nMPLAYER=”\/usr\/local\/bin\/mplayer”
\nPARAMS=”-nosound -noaspect -vop scale=256:256,crop=480:480:80:0 -vo glstream:dxt3:lzf:verbose:file=”
\nfor i in *.avi; do
\nINPUT=$i
\nOUTPUT=${i%%.avi}_box.dxt3_lzf
\n$MPLAYER $PARAMS$OUTPUT $INPUT
\ndone
\nNote: The target frame size (set with ft.start() in the LUA script) and the texture size (set by the scale option of mplayer) should match for best performance during playback.<\/p>\n

Smplayer2 LUA API<\/h6>\n

Global API<\/u>
\nNamespace<\/strong>
\nglobals.xyz
\nsmplayer.xyz
\nProperties<\/strong>
\nwidth (Get, Set)
\nheight (Get, Set)
\nscreens (Get, Set)
\ncursor (Get, Set)
\noutline (Get, Set)
\ndraw (Get, Set)
\nFunctions<\/strong>
\ncreate() : create object and make it global (automatically called by smplayer2)
\nlist() : displays current smplayer configuration for debugging purposes
\nfloat[8] = getCorners(screen) : returns current display positions of screens
\nsetCorners(screen, float[8]) : set display positions of screens for flatfield calibration
\nquit() : quit game loop and exit smplayer<\/p>\n

Frametable API<\/u>
\nNamespace<\/strong>
\nft.xyz
\nProperties<\/strong>
\nscreens (get, set) : the screen to place the frame onto
\ntw (get) : the texture width; power of two
\nth (get) : the texture height; power of two
\nr (get, set) : red filter, range=[0.0 \u2013 1.0], default=1.0
\ng (get, set) : green filter, range=[0.0 \u2013 1.0], default=1.0
\nb (get, set) : blue filter, range=[0.0 \u2013 1.0], default=1.0
\na (get, set) : transparency of frame, range=[0.0 \u2013 1.0], default=1.0
\nrate (get, set) : update framerate in Hz
\nx (get, set) : top right X position on logical screen, range=[0.0 \u2013 1.0]
\ny (get, set) : top right Y position on logical screen, range=[0.0 \u2013 1.0]
\nw (get, set) : width on logical screen, range=[0.0 \u2013 1.0]
\nh (get, set) : height on logical screen, range=[0.0 \u2013 1.0]
\nFunctions<\/strong>
\nstart(frame_id, texture_width, texture_height, framerate_hz) : creates new frame object in frametable
\nlist() : lists all frames in frametable for debugging purposes<\/p>\n

stop(frame_id) : deletes frame from frametable
\nclone(source_frame_id, target_frame_id) : sets the content to be cloned from existing frame
\nunclone(frame_id) : disables cloning for a frame
\ndelay(delay_milliseconds) : do a millisecond delay
\ncolor(frame_id, r, g, b, a) : sets the color filters and transparency
\nrate(frame_id, rate) : sets the frame rate for updates
\nfloat[8] = getCorners(frame_id) gets the current corner coordinates in the logical coordinate system [0,0 \u2013 1,1]
\nsetCorners(frame_id, float[8]) : sets the current corner coordinates coordinates in the logical coordinate system [0,0 \u2013 1,1]
\nwarp(frame_id, x1,y1,x2,y2,x3,y3,x4,y4) : sets the current corner coordinates coordinates in the logical coordinate system [0,0 \u2013 1,1]; equivalent to setCorners but different call interface
\ncrop(frame_id, float[8]) : : sets new source texture coordinates; used for cropping or enlarging textures
\nblit(frame_id, media_id, x, y, blendFlag) : transfer mediatable content into frame
\ns3t_read(frame_id, fileHandle) : read texture into frame; sync IO
\ns3tc_tioread(frame_id, tioFileHandle) : read texture into frame; multithreaded async IO
\nsave(frame_id, filename) : save current frame to a file for debugging
\nupdate(frame_id) : mark frame for texture update when rate interval expires
\nvncopen(frame_id, serverHost, serverPort, serverEncoding, serverPassword) : open connection to VNC server
\nvncupdate(frame_id) : update frame from vnc connection
\nvncclose(frame_id) : close vnc connection
\ngrabopen(frame_id, device, width, height, channel, mode, frequencyRegion, frequencyIndex) : open framegrabber device<\/p>\n

grabupdate(frame_id, deinterlaceFlag) : update frame from framegrabber device
\ngrabclose(frame_id) : close framegrabber connection<\/p>\n

Mediatable API
\n<\/u>Namespace<\/strong>
\nmt.xyz
\nProperties<\/strong>
\nw (Get) : width of media surface
\nh (Get) : heigh of media surface
\ncx (Get, Set) : top right X position of clipping rectangle of media surface
\ncy (Get, Set) : top right Y position of clipping rectangle of media surface
\ncw (Get, Set) : width of clipping rectangle of media surface
\nch (Get, Set) : height of clipping rectangle of media surface
\nFunctions<\/strong>
\nimage(filename) : create new media surface by load image from file
\nclone(media_id) : create new media surface from existing media
\ncreate(width, height) : create new blank media surface
\ntext(text, fontfile, fontsize, fontstyle, color) : create new media surface from text
\nrotozoom(media_id, angle, zoomFactor) : create new media surface by rotating and zooming existing media surface
\nlist() : display all media surfaces
\nclear(media_id, alpha) : clear media surface to particular alpha value
\nfill(media_id, color) : fill media surface with color
\nwidth(media_id, w) : set width of media surface
\nheight(media_id, h) : set height of media surface<\/p>\n

blit(source_media_id, target_media_id, x, y, useAlphaFlag) : blit source media surface to target media surface at specified position optionally using alpha blending<\/p>\n

clip(media_id, x,y,w,h) : set clipping rectangle for drawing operations on media surface<\/p>\n

unclip(media_id) : clear clipping rectangle of media surface<\/p>\n

SDL_gfx Mapped Functions<\/strong><\/p>\n

pixel(media_id, x, y, color) : draw pixel
\nhline(media_id, x1, x2, y, color) : draw horizontal line
\nvline(media_id, x, y1, y2, color) : draw vertical line
\nrectangle(media_id, x1,y1,x2,y2, color) : draw rectangle
\nbox(media_id, x1,y1,x2,y2, color) : draw filled rectangle
\nline(media_id, x1,y1,x2,y2, color) : draw line
\naaline(media_id, x1,y1,x2,y2, color) : draw antialiased line
\ncircle(media_id, x,y,r, color) : draw circle
\naacircle(media_id, x,y,r, color) : draw antialiased circle
\nfilledcircle(media_id, x,y,r, color) : draw filled circle
\nellipse(media_id, x,y,rx,ry, color) : draw ellipse
\naaellipse(media_id, x,y,rx,ry, color) : draw antialiased ellipse
\nfilledellipse(media_id, x,y,rx,ry, color) : draw filled ellipse
\npie(media_id, x,y,rad,start,end, color) : draw pie
\nfilledpie(media_id, x,y,rad,start,end, color) : draw filled pie
\ntrigon(media_id, x1,y1,x2,y2,x3,y3, color) : draw triangle
\naatrigon(media_id, x1,y1,x2,y2,x3,y3, color) : draw antialiased triangle
\nfilledtrigon(media_id, x1,y1,x2,y2,x3,y3, color) : draw filled triangle
\npolygon() : not implemented
\naapolygon() : not implemented
\nfilledpolygon() : not implemented
\nbezier() : not implemented
\nstring(media_id, x,y, text, color) : draw text in 8×8 fixed width font to media surface<\/p>\n","protected":false},"excerpt":{"rendered":"

This is the source archive for smplayer a Linux+OpenGL based LUA scriptable mediaplayer that supports HD video playback and custom digital IO controls; it was used as Digital Retina media art installation by Jill Scott in Version 2.0.<\/p>\n","protected":false},"author":1,"featured_media":122,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[22],"tags":[36,61,62,31,35],"class_list":["post-75","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-c","tag-digital-retina","tag-jill-scott","tag-media-art","tag-opengl"],"_links":{"self":[{"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/posts\/75","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/comments?post=75"}],"version-history":[{"count":5,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/posts\/75\/revisions"}],"predecessor-version":[{"id":121,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/posts\/75\/revisions\/121"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/media\/122"}],"wp:attachment":[{"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/media?parent=75"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/categories?post=75"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ferzkopp.net\/wordpress\/wp-json\/wp\/v2\/tags?post=75"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}