Marshall Breeding
Report of the Director for Innovative Technologies and Research
August 2006
Vanderbilt Television News Archive
A statistical summary of the activities of the Vanderbilt Television News Archive is available here.
A cumulative statistical summary of the Archive's activities for 2006 calendar year is available here.
NEH Project
Work continues on the NEH grant-funded project to digitize the Archive's collection of news specials.
Archiving of Local News
Discussions continue regarding a possible partnership between the Nashville Public Library, WSMV television, and the Vanderbilt Television News Archive regarding a possible project to digitize the archive of the news programs of the local station and to perform ongoing off-air recording. Marshall drafted a project proposal that describes the major components of the project and costs involved. This proposal is currently being reviewed and revised working toward selecting possible funding opportunities.
The Archive has begun producing copies of the 6PM Channel 4 News program on DVD. We anticipate being able to use the transcripts produced from the closed caption file as a finding aid for these programs since producing abstracts for these programs is not current feasible.
Digital Recording System
This month, Marshall began rebuilding the 24-hour digital recording system according to the new design developed in previous months. Some of the changes in the new design include:
- New software for both the Hauppauge TV card that runs on each of the recorders (pvr350.exe). This software previously only produced and analog audio and video signal that was fed into the OptiBase MPEG encoding card, via the XBOB that produced the Network/Time/Date overlay. This software now also produces its own MPEG-2 recording (without the overlay) and a file of raw closed caption data. This software is written in C++ using the Microsoft Visual Studio .NET C++ development environment with the DirectX software development kit and the DirectX filters from Hauppauge. File management routines developed for the Hauppauge card software were employed to make file name assignments and to switch to a new file every hour for both the MPEG-2 video file and the raw closed caption file. Changes were made in the ocbc.cc module to accommodate the migration from DB/TextWorks to MySQL. Once a minute this script updates its record in tvnprocessmonitor table to verify its activity.
- New software for the Optibase MPEG-2 encoding card (encode.exe). The software was modified to use the MySQL database and to add a type field when registering an MPEG to distinguish this card's activity from the Hauppauge card. This software is produced with the Microsoft Visual .NET C++ development environment in conjunction with the Software Development Kit provided by Hauppauge. Once a minute this script updates its record in tvnprocessmonitor table to verify its activity.
Library37 serves as the control system for the recording cluster. While the previous version of the recording system was based on DB/TextWorks, ODBC, and Perl, this version relies on MySQL, ODBC, and Perl. The following MySQL tables comprise the diglib database for the system:
- tvnrecordedprograms. Each file recorded is registered in this table by either encode.exe or pvr350.exe. The initial registration includes the source recorder, the file name, the status (Recording), the Network, and the BeginTime. Once the recorder finishes the file, the status changes to Finished and an EndTime is recorded. The Analyze perl script on the control server polls for records with the status Finished to enhance the record with additional metadata and to assign a deletion priority.
- tvnrequestedprograms. This table consists of program listings for each network covered by the recording cluster, including values for the program title, day of week, time of day, and a numerical value according to its potential news interest.
- tvnprocessmonitor. This table holds the operational status of each component of the system, including the encoders, TV cards, and each of the perl scripts that manage a component of the system.
Several perl scripts run on the control server, including:
- tvn-recording-analyze.pl. This script polls the tvnrecordedprograms for new records with the status "Finished," representing programs that have completed the recording phase. For each finished program, the script polls the tvnrequestedprograms table according to day of week, time, and network to check for a corresponding listing. If it finds one, it transfers the ProgramTitle, and calculates a DeletionPriority by adding the assigned InterestLevel in tvnrequestedprograms to the numerical value of the Gregorian Date. For each of the programs recorded by the Hauppauge PVR-350, as triggered by the -tv suffix on the file, the script initiates the creation of a transcript. With each iteration, this script updates its record in tvnprocessmonitor table to verify its activity.
- tvn-recording-autodelete.pl manages the process of file deletion so that none of the disk volumes run out of space. Based on a configurable threshold, currently 80%, this script monitors the space available on each volume and automatically deletes the files with the least potential news interest first. The script will not delete files that are less than 72 hours old. This script also removes files manually selected for deletion.
- tvn-recording-process-monitor.pl, based on the tvnprocessmonitor table, monitors the operation of each component of the system. If any of the components fails to register activity within an assigned threshold, this script takes action to reinstate it. For the perl scripts on the control server, it kills any existing instance of that process and re-launches the script; for problems with the recording servers, it issues a directive to restart the corresponding server.
New hardware was obtained for the new control server. A Dell PowerEdge 2850, known as Library37, will manage the other devices in the cluster. This server has 1.2TB of internal disk storage and will connect to 2-AX-100 storage arrays with about 4TB each.
The disk space on each of the existing Dell PowerEdge servers was doubled by replacing both 250GB drives with 500GB drives. Since each recorder now produces two MPEG files, its was necessary to double the storage capacity to retain the same level of program content. Each recorder now has 1TB of SATA disk storage.
Once the drives are replaced, the system is rebuilt from scratch: Installation of Windows Server 2003, Application of all accumulated security patches, installation of drivers for USB Baudio card, encoding card, and tv card, installation of the custom software for the Optibase and Hauppauge cards, configuration of the video disk volumes, creation of user accounts and applying security policies. The transition from the old recording system to the new one involves moving the systems one at a time. In August we moved Lib30,Lib31,Lib32,Lib35 and added a new system, Lib38. Completion of the transition is planned by the end of September.
Primo
Marshall participated in the day-long meetings with Ex Libris staff. On Aug 14 as part of the Primo Open House event for library staff, Marshall gave an overview of Primo and its underlying concepts.
Meetings
Marshall participated in the Library Management Council retreat, and other regular committee meetings (Strategy and Planning Council, Metadata Committee, TV News Staff) and participated in the quarterly New Employee Welcome Session
Professional Activities
Marshall conducted a day-long SOLINET workshop on Wireless Technologies in Libraries at the Houston County Library in Georgia on Aug 18.
As a contributing editor for ALA TechSource Smart Libraries Newsletter, Marshall wrote articles on the the appointment of a new general manager for NISO, the launch of the Open Source Evergreen ILS for the Georgia Pines consortium, and the acquisition of DiMeMa, producers of CONTENTdm by OCLC.
Marshall's regular column appeared in Computers in Libraries.