-
The reason for this is (probably) that python lists are in fact clever resizable arrays, while OCaml lists are a chain of cons cells.
Avoid the list completely; in python you would want to use the file's iterator anyway (for line in file("x.y"):...). There is a List.iter and a String.iter function, why not build a File_iter to do the job?
Hmmm. I think I need to experiment a little with this... ;)
-
After thinking about it a little, here is what I came up with:
Since for a parser you'll probably want to save some state, how about the following?
let bufsize = 4096;;
let fold_chan func chan base =
let buf = String.create bufsize in
let rec fold_loop func chan acc = begin
String.iter (fun c -> acc := func c !acc) buf;
if (input chan buf 0 bufsize) = bufsize then
fold_loop func chan acc else !acc
end in
let acc = ref base in fold_loop func chan acc;;
My test case: Get the ascii value of the 'highest' character of a file
let inc = open_in Sys.argv.(1);;
print_int (fold_chan (fun x y ->
let ix = int_of_char x in
if ix < y then y else ix) inc 0);;
close_in inc;;
print_newline ();;
I did not take the time to optimize, but it should run quite good.
The python version:
import sys
import time
start = time.time()
x = 0
for c in file(sys.argv[1]).read():
oc = ord(c)
if oc > x: x = oc
print x
stop = time.time()
print "%fs"%(stop-start)
The ocaml version is faster than the py version, at least on my PC.
$ python foldchan.py some_file.dat
252
0.060170s
$ ocaml foldchan.ml some_file.dat
252
0.03s
Yay for the OCaml! ;)
-
Ah well, that's why I dislike languages with Significiant Whitespace (TM)...you know the py version does not work like it looks in HTML...but I am too lazy to change all spaces to s...
-
Oops. Made a mistake in the OCaml version: the input must be evaluated before the String.iter:
let bufsize = 4096;;
let buf = String.create bufsize;;
let fold_chan func chan base =
let rec fold_loop func chan acc =
& nbsp; let chars = (input chan buf 0 bufsize) in
(String.iter (fun c -> acc := func c !acc) buf;
if chars < bufsize then !acc
else fold_loop func chan acc) in
fold_loop func chan (ref base);;
let start = Sys.time();;
let inc = open_in Sys.argv.(1);;
print_int (fold_chan (fun x y ->
let ix = int_of_char x in
if ix < y then y else ix) inc 0);;
close_in inc;;
Printf.printf "\n%fs\n" (Sys.time() -. start);;
Compare with (also much updated) python:
import sys
import time
def foldchan(func, stream, base):
return reduce(func, stream.read(), base)
start = time.time()
print foldchan(lambda x, y: max(x, ord(y)), file(sys.argv[1]), 0)
stop = time.time()
print "%fs"%(stop-start)
The python version is now more reusable without being larger (thanks to the builtin reduce). It however is also somewhat slower.
It bugged me that the python version is so much smaller (wc: ml: 19 lines, 96 words, 550 chars, py: 11 lines, 28 words, 235 chars) until I realized that it will chocke on really big files (as read() wants to slurp it into RAM) and to create a meaningful equivalent version you would need probably as much code as in OCaml. And the execution speed tells it all:
$ python foldchan.py some.mp3
255
9.748185s
$ ocamlopt -o foldchan foldchan.ml && ./foldchan some.mp3
255
0.130000s
-
Nice. I knew that there must be a way to make OCaml perform better, I just didn't know how to. Thanks!
-
For what it's worth, just for fun I decided to spend about a minute rewriting the Python version of this to:
import sys
import time
def foldchan(func, stream, base):
return reduce(func, stream.read(), base)
def maxChar(f):
return ord(max([max([ch for ch in fline]) for fline in f ]))
f = file(sys.argv[1], 'r')
start = time.time()
print foldchan(lambda x, y: max(x, ord(y)), f, 0)
stop = time.time()
print "orig: %fs"%(stop-start)
f.close()
f = file(sys.argv[1], 'r')
start = time.time()
print maxChar(f)
stop = time.time()
print "listComp: %fs"%(stop-start)
f.close()
I got this result:
$ python maxChar.py some.mp3
255
orig: 0.266000s
255
listComp: 0.015000s
So, even within Python we can get x17 speedup, with fewer lines of code, too. It's still looks like about 4 times slower than OCaml, though.
-
JO'N: I knew you could do that (and you might save memory with python 2.4s generator expressions, at the expense of just a small amount of cycles. But: The max function was only one example.
In the ocaml version you can cook up a completely different algorithm that folds all characters without touching the fold_chan function, while in py you'd have to recode your algorithm. Example: Say we'd want to count the number of 'x's then our code would be:
def xcount(f):
return sum(l.count("x") for l in f)
# add [] in the sum if you live before 2.4 ;)
Whoa. This is a very simple case, yet the schema:
def {func_name}(f):
return {cast_func}{func}([{func}(c) for c in l]) for l in f])
already fails, because sum is not l.count. We now need to know that counts have to be combined by sum(). What if we wanted to have a histogram? Or the average occurrence of "x"? We would a) need to join the stuff into one string, requiring a huge amount of cycles and memory or b) be more clever than we need to be everytime we write a function like this.
In ocaml, we have solved this problem once and for all. I get away with being only this clever. Functional programming promotes reuse (IMHO more than OOP).
Oh and by the way I can shave off one line-of-code from my script by coding the test function as such:
print_int(int_of_char(fold_chan (fun x y ->
if x < y then y else x) inc '\000'));;
Since the int_of_char was in the folding function before, this *might* even have been faster, if it wasn't for ocamlopt being so clever that it allows me to be sloppy and still blow my socks off... ;)
python wins anyway, on its readable and consistent syntax plus the huge amount of library code. If only the stdlib for OCaml was a little bigger and a little more object-oriented...
-
I noticed that the OCaml doesn't seem to work so well on non-text files, or even on text files, where it seems to find unexpected EOFs and such. (At least on Windows.) Replacing open_in with open_in_bin fixes this, but the OCaml code appears to be slower now (possibly because it didn't read the whole file beforehand?).