Stefan
|
Let’s see… for starters, we’ll need to agree on database structure. It doesn’t have to be that difficult, I think.
Table soundClip
int user_id // The user id from the forum, to credit the contributor
string description // A short description provided by the user
string license-type // I think providing the Creative Commons ones will be enough.
//The following are completed by the staff member
url filename // The internal filename. The compressed files will be named the same, but placed in a different directory. Perhaps a different directory for each category?
string category // Where does it fall?
string subcategory // Not sure if this is needed, depends on how large the directory gets and how we categorize.
boolean is_public
int times_downloaded
Do we need any others? It’ll get a bit awkward for mass uploads, though. Will think about that. Perhaps a “copy from previous entry” button on a form with multiple entries will be nice.
I want the curator to be able to dynamically create categories. Perhaps a category needs to be a database table. Yes, why not make categories and subcategories the same, such as in http://www.dmoz.org/World/F%c3%b8royskt/List/ where there are sites listed, but also subcats available for more specific sounds. Mmmm, this is going to be nice!
I think we should limit download access to brickfilms.com members. Otherwise the bandwidth requirements might get awkward as soon as fanfilmers get word of it.
For the automated conversion (better invest once than cause headaches on the moderator), wavetools looks like a good start. I’m sure a command-line Ogg encoder is readily available, but will have to research what’s in the Vorbis Tools package. Something similar is bound to exist for Real as well.
Stefan.
P.S. I think I have time next sunday. Maybe Saturday night even.
|
RevMen
|
Perhaps submission doesn’t need to be by database. Maybe we can do it in a forum or via email or PM. I sort of envision the curator of the library assembling a well-rounded collection on his own instead of simply taking every submission that comes along. I think a submission form, like there is for our film directory, would only encourage submission of lots of less-than-savory material.
|
Blunty
|
“RevMen” wrote: Perhaps submission doesn’t need to be by database. Maybe we can do it in a forum or via email or PM. I sort of envision the curator of the library assembling a well-rounded collection on his own instead of simply taking every submission that comes along. I think a submission form, like there is for our film directory, would only encourage submission of lots of less-than-savory material.
I’m with Rev, I had envisioned mush the same thing. collecting and constructing a “base” libruary but also adding to that by taking submissions though a form or PM…
but the access to the sound directory would opperate much the same as our film directory.
|
Stefan
|
Another (even better) idea: the curator uploads the files to a certain “temp” directory, then logs into the curator’s webpage. This page automatically scans the temp directory, and creates a form element for every file present. The curator completes info, submits, and the submission engine will convert/compress the files, create database entries and make them available to the public.
Stefan.
|
Blunty
|
now that’s an intersting idea Stefan.
|
RevMen
|
I’d rather not have compression be automatically done on the server. I’d actually rather have the curator do the compression, to make sure quality is acceptable. That is an interesting idea, though.
|
Stefan
|
I was thinking about doing different versions. Perhaps have the highest quality version prepared by the curator, and the scripts generating low-quality previews (11kHz, 8 bit, lossy compression). The user is presented with a “preview” button and a “download” button.
Also, if the curator uploads uncompressed WAVs, he can first run ALL compression automagically, and the curator checks and manually re-encodes problematic entries. If we are generous with standard encoding parameters, these cases can be minimal.
Anyway, I think we’ll need the automatic process for at least the preview file, so I’ll just go ahead and program it. Besides, it’s no fun when it’s not a challenge 
Stefan.
|
RevMen
|
Anyway, I think we’ll need the automatic process for at least the preview file, so I’ll just go ahead and program it. Besides, it’s no fun when it’s not a challenge
I don’t think that you think we actually need it, I think you just want to do it for the challenge. 
Yeah, having a preview automatically generated would be very cool.
|
LowweeK
|
Excellent ! Stefan, your ideas are very good.
Another one : is a public FTP with incoming directory doable ? recommended ? a good thing ?
Aside from your excellent criterias, a got plenty of others.
Some might be technical, like : length, quality, format, date (perhaps automatically generated ?)
Some might concern the production : gear used (mic, preamp), indoor/outdoor, engineered, effects, etc.
For the categories, I think we should have two different groups : FX and music, because the description is totally different…
For FX, the categories mostly are a functional description of the sound (animal, vehicles,…). Description provides a detailed description : like “outdoor little river with distant dogs barking”. As an owner of some collections, I got some views on the point.
For the music, categories are of course, the genre, and some additionnal criterias like “dark”, “weird”, “powerful”, “airy”. The description can provide some details like the instruments, singer, recommended use (“perfect for horror movie”).
Too bad I did miss a conference about Creative Commons some days ago… 
|
Stefan
|
Another one : is a public FTP with incoming directory doable ? recommended ? a good thing ?
From the sounds I hear from Blunty and RevMen, I think we can set up temporary accounts for people who want to upload collections.
About licenses. I think we need the sounds to be public domain – it’s impossible for a filmmaker to credit each sound used. For music, something like the Sampling Plus license seems appropriate.
Stefan.
|
Stefan
|
I’d like to install the following packages on the server (in this order):
libogg
libvorbis
libmad
SoX
flac
Source code and rpms at following sites:
http://www.vorbis.com/download_unix_1.0.1.psp
http://www.underbit.com/products/mad/
http://sox.sourceforge.net/
http://flac.sourceforge.net/download.html
libmad is mostly meant to be able to re-encode source files. I intend to feed the uploaded files to SoX (possibly after unzipping). I’m going to use something like system() to run all these things.
I think I won’t go trying to encode MP3, but for the music section we might consider not converting MP3-format songs that are uploaded – or just require our artists to use FLAC or OGG.
Workflow:
1) User uploads .zip or .snd file (where .snd in {.wav, .flac, .ogg, .mp3, …}). File is copied and unzipped into the queue directory. User does uploading through php page.
2) Curator opens queue page, sees files in queue, can mark some as “do not recompress”, completes information if necessary and pushes button. This causes the system for each file to
- copy the file from queue to library if it doesn’t have to be processed
- recompress the file from queue into library otherwise
- compress file to ultra-low quality file format for previewing (I’m thinking 8-bit, 11kHz, Ogg or wav)
- enter the file info into the database.
3) Curator can also give specific users permission to access the upload page, and remove these permissions when the uploading is done.
I’m hoping to determine at least length of the sound automatically, but this might prove tricky.
I’d like to write a log containing all output from the external commands, so that it’ll be possible to track errors. Perhaps a manual submission form to correct failures is useful. The curator can view and empty this log.
Finally, some caution is needed to prevent existing files to be overwritten by files that are later uploaded and carry an identical name. Probably an automatically incrementing counter will be suffixed to the filename.
Now it’s time to start studying – more on this later!
Stefan.
|
Blunty
|
That’s all very clever stuff Stefan, but it seems unnessisarily complex.
I’d prefer to have the files checked and encoded locally before uploading the “ready” files to the server. That way a lot of extra load for the server is saved, quality is assured, then the only encoding the server need do is a simple lo-fi sample for preview purposes.
|
Stefan
|
That’s ok. In that case, we’ll have
1) An upload feature for occasional contributors. They simply stash the file(s) on the server for you to download. You can download them and then delete them. No unzipping in this case.
2) A database addition + lo-fi conversion page for you. I assume using your (secure) ftp access rights allows resuming etc. so that would be preferable for your uploading needs.
3) A rights management tool to give users temporary access to the server.
Do you want me to go through the trouble of extracting “ID3 and friends” info from the files, or are you content with pre-specifying the length only – if even that is feasible – ?
Stefan.
|
Stefan
|
The first part of the system, the uploading interface, is done. Bluntman, you can start collecting files now. Just go to uploadform.php, add the user_id’s of the users you want to receive files from and direct them there. People with only upload access will see everything up to but excluding the Curator part, the Curator will see the Curator part as well, and other users see an empty page save for the title.
Apart from Bluntman, I also made Z, RevMen and myself curator – just to show off how cool a page that is The “Sounduploader” user management system can easily be rewritten to do patrons, programmers, reviewers and other custom privileges.
I uploaded a 35MB file, which failed. Not sure what the limits are. I put 250 MB in the form’s code, but there is some limit in php.ini or php.conf as well.
EDIT: Yep. /etc/php.ini reports 2M. Can I up this to 250M, RevMen?
—
Now for the real system. From the forum thread it appears as if we’re having .ogg, .mp3 and .flac, and additionally a lo-fi preview file in – most likely – .mp3 format. I still believe that doing conversion on the server after being presented with the .flac or .wav file is the way to go – consistent bitrate+settings = consistent quality. We’ll be quite generous on the high-quality side. The server will be converting anyway, so why not just go for it all the way?
And Blunty, are you still on 56k? Otherwise I’d suggest simply uploading uncompressed .wav files, for optimum quality.
Stefan.
|
RevMen
|
Yep. /etc/php.ini reports 2M. Can I up this to 250M, RevMen?
Yes, of course, do what you need to do.
|
Connect with Us