Archives for December, 2013

Scaling *up* Hadoop for under-100-GB jobs

Scaling *up* Hadoop for under-100 GB jobs “Nobody ever got fired for buying a cluster” – A very interesting Microsoft research paper implying: Most common huge jobs are still under 100GB (yes, including the elephant friendly Facebook) Addressing the “problematic” issue of “when and how to distribute with Hadoop”, if any? Addressing the cost efficiency […]

When Lucene met Hadoop. The Blur project

When Lucene met Hadoop. The Blur project So expected. Still incubating, but looking good 🙂 Though, only a complete Solr-over-Hadoop solution would close the deal, in my opinion

File upload – simple multipart request stream parser, using Regex

(In case your text file contains line starting exactly with the “matched” ones below – simply adapt your Regex) private static List<string> fileUploadRequestParser(Stream stream) { //—————————–111111111111111 //Content-Disposition: form-data; name=”file”; filename=”data.txt” //Content-Type: text/plain //… //… //—————————–111111111111111 //Content-Disposition: form-data; name=”submit” //Submit //—————————–111111111111111– List<String> lstLines = new List<string>(); TextReader textReader = new StreamReader(stream); string sLine = textReader.ReadLine(); Regex […]