The rendering of the lgm posters is split into two asyncronous parts, because the actual poster generator involves some stuff that no administrator wants to have running on a stable web server. Part 1 is a simple input interface, that does nothing more than writing plain text files entered via a html textarea. This runs on the lgm server. Part 2 synchronizes with the online part, generates posters and uploads new files. This runs on a machine in the our studio.
The script starts by defining some general variables and storing information about the current/local voice files.
TMPDIR=tmp PDFDIR=o/__ VOICEDIR=o/free/voiceREMOTELIST=http://libregraphicsmeeting.org/2013/r+w/list.php
The current, local number of voice files is stored (VNUMLOCALE=`ls $VOICEDIR/*.txt | wc -l`
) and I start a list with 40 random voice files, that will be used together with new files downloaded from the server.
VNUMLOCALE=`ls $VOICEDIR/*.txt | wc -l`ls -t $VOICEDIR/*.txt | \ rev | cut -d "/" -f 1 | rev | \ shuf -n 40 > q+a.list
To be able to differentiate new from old voice files I do the following:
ls $VOICEDIR/*.txt | rev | cut -d "/" -f 1 | rev | sed ':a;N;$!ba;s/\n/|/g'
lists all current voice files separated by newlines (:a;N;$!ba;s/\n
) and replaces the newlines with a |
. This string that is used later on by egrep
as a filter (e.g. egrep -v "foo.txt|bar.txt|montana.txt
).
XGREP=`ls $VOICEDIR/*.txt | \ rev | cut -d "/" -f 1 | rev | \ sed ':a;N;$!ba;s/\n/|/g'`
For downloading new voice files from the server I use wget
. Because directory listing is disabled on the lgm server we use a php file that lists all txt files in the voice directory. I download this list (wget -O $LIST $REMOTELIST
) to use it in the next step as input file for wget (wget -r -np -nH --no-directories -N -S -i $LIST
).
# -------------------------------------------------------------------------------------- # # UPDATE QUESTIONS & ANSWERS # -------------------------------------------------------------------------------------- #REMOTEDIR=http://www.forkable.eu/generators/r+w/i/free/voice/ LOCALEDIR=$VOICEDIRcd $LOCALEDIRLIST=voices.list wget -O $LIST $REMOTELISTwget -r -np -nH --no-directories -N -S -i $LISTcd -rm $LOCALEDIR/$LIST
After downloading I can calculate the number of remote voice files (which are now local): VNUMREMOTE=`ls $VOICEDIR/*.txt | wc -l`
. By filtering out all voice files which have been there before the synchronisation (ls -t o/free/voice/*.txt | rev | cut -d "/" -f 1 | rev | egrep -v "$XGREP"
) I get a list of all new voice files, which are added to the previously started list. I use this combination of old and new voice files, because I decided to generate also posters from already existing voice files on each synchronisation run (NUM2GENERATE=`expr $VNUMREMOTE - $VNUMLOCALE + 1`
).
# -------------------------------------------------------------------------------------- #VNUMREMOTE=`ls $VOICEDIR/*.txt | wc -l`ls -t o/free/voice/*.txt | rev | cut -d "/" -f 1 | rev | \ egrep -v "$XGREP" >> q+a.list# -------------------------------------------------------------------------------------- #NUM2GENERATE=`expr $VNUMREMOTE - $VNUMLOCALE + 1`if [ $VNUMREMOTE -gt $VNUMLOCALE ] then # -------------------------------------------------------------------------------------- ## -------------------------------------------------------------------------------------- # # CREATE NEW POSTERS # # -------------------------------------------------------------------------------------- #CNT=1 ; while [ $CNT -le $NUM2GENERATE ]; doSELECTID=`tac q+a.list | \ rev | cut -d "_" -f 2 | cut -d "/" -f 1 | rev | \ awk ' !x[$0]++' | head -n $CNT | tail -1` SELECTQ=`ls -t i/free/voice/*_Q.txt | grep $SELECTID | head -n 1`./do.sh $SELECTQ CNT=`expr $CNT + 1`done
Finally I upload the generated stuff on the webserver using ftp.
# -------------------------------------------------------------------------------------- # # UPLOAD NEW FILES # -------------------------------------------------------------------------------------- ## GENERATE FTP COMMANDS ---------------------------------------------------------------- #ACCESS=`cat ~/.forkable/ftp.input`echo $ACCESS > ftp.tmp
for PDF in `ls -t ${PDFDIR}/*.pdf | head -n $NUM2UPLOAD` do echo "put $PDF "`basename $PDF` >> ftp.tmp donefor GIF in `ls -t ${PDFDIR}/*.gif | head -n $NUM2UPLOAD` do echo "put $GIF "`basename $GIF` >> ftp.tmp doneecho "put $ALLHTML "`basename $ALLHTML` >> ftp.tmp echo "bye" >> ftp.tmpmv ftp.tmp ftp.input# UPLOAD VIA FTP ----------------------------------------------------------------------- #ftp -n forkable.eu < ftp.input rm ftp.input# -------------------------------------------------------------------------------------- #
else echo "-> NOTHING NEW" fi rm q+a.listexit 0;
The voice files are saved and versionized in a git repository, because some spelling correction and spam deletion had to be done. Thanks to Ale Rimoldi for taking care for the setup on the lgm server and the github synchronization.