Forums  > Software  > Need an excuse to learn Fortran  
     
Page 1 of 1
Display using:  

Maggette


Total Posts: 1056
Joined: Jun 2007
 
Posted: 2018-04-03 08:39
I do work as a data scientist, data/software engineer and do some operations research stuff (simulations, optimization).

So far my workhorse is python. Almost 50% of my work is done in python. If things have to speed up there, I use python numba.

About 40% of my work is done in Scala. This is because lots of the Apache Foundation big data stack is written in Scala or interfaces well with it (Spark, Flink, Kafka,Kudu,HBase) and because of Akka. Scala is a compiled language and reasonable fast for most applications-

My problem is now if I have to speed up in the JVM world, I miss something like using numba in python. So if I need to implement something that I call often and needs to be fast(say something like custom made meta heuristic) I have no tool for that.

So I thought if I could get some practice writing Fortran and try to call the routines from the JVM using JNA (https://en.wikipedia.org/wiki/Java_Native_Access).

Am I the only person who think that makes sense?

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

Patrik
Founding Member

Total Posts: 1355
Joined: Mar 2004
 
Posted: 2018-04-04 09:52
If you want an excuse then why not Smiley

I would have thought that c/c++/rust using BLAS/LAPACK/etc may involve less yak shaving if you have some background, without any meaningful difference to doing it all in fortran.

A run-fast-from-jvm project in clojure I like - for inspiration: https://neanderthal.uncomplicate.org/

Capital Structure Demolition LLC Radiation

EspressoLover


Total Posts: 334
Joined: Jan 2015
 
Posted: 2018-04-04 14:57
+1 to Patrik's suggestion

Fortran can definitely have higher numerical performance than C. (Or at least naive C not optimized by some guru.) But most everything where Fortran outperforms already has pre-existing libraries like BLAS. Fortran is pretty terrible from a language standpoint, and writing it is definitely a lot more painful than even idiomatic C. Before proceeding, I'd also profile to verify the assumption that the performance bottleneck actually lives in the numerical computations.

Another approach might be to consider just offloading the foreign language calls as a REST API, and just run as a separate process. Obviously this doesn't work if you're calling a thousand times a second. But if you can refactor to batch the calls, then network latency becomes de minims as it's amortized.

Good questions outrank easy answers. -Paul Samuelson

Maggette


Total Posts: 1056
Joined: Jun 2007
 
Posted: 2018-04-05 08:36
Thanks to both of you

Like I said. At present I have no such problems. Flink/Akka utilizing Scala and parallelization has solved every problem I had in timly manner. So far sub-second or double digit milliseconds speed in respect of lattency for real time applications was everything I needed to provide and the problems weren't that computional expensive. Similar for all the batch stuff I had to do.

But I did solve stuff in python, using numba and naive parrallization that I am not sure I could do as fast in Scala on a cluster in an Akka/Flink/Spark application.

Regarding the learning proccess: I only wrote some small (and really bad) C programs so far. And I do like array based stuff, like q, numpy and Matlab..so far, I didn't find Fortran that much harder to read than C. And modern Fortran has an natural model for parallel computation I do not see in C or C++.

On the other hand Neanderthal looks very nice. This is reassuring!







Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

urnash


Total Posts: 564
Joined: Sep 2006
 
Posted: 2018-04-10 20:17
I've been spammed a lot recently with adverts for https://www.manning.com/books/modern-fortran. Might be useful.

Jim Simmons is finalizing a c. 1000 page tome on Renaissance trading methods and is looking for a publisher. He was turned down by Wiley. They said he's taxing the attention span of their readers -- LongTheta

Maggette


Total Posts: 1056
Joined: Jun 2007
 
Posted: 2018-04-11 06:54
Thx urnash.

Actually I was considering Fortran a while ago but let it slide, until I got the exact same spam mails too and this sparked my interest again:).

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

katastrofa


Total Posts: 458
Joined: Jul 2008
 
Posted: 2018-04-14 12:29
I wonder if they touch on the subject of automatic differentiation in Fortran.

jslade


Total Posts: 1136
Joined: Feb 2007
 
Posted: 2018-04-19 20:26
I've done this; it's usually worth it, but you can become IO bound copying to-from the JVM. Some people do the allocation on the JVM heap and pass that to Fortran, but you're probably not going to do that.

"Learning, n. The kind of ignorance distinguishing the studious."

Maggette


Total Posts: 1056
Joined: Jun 2007
 
Posted: 2018-04-20 11:37
Thx. Could you elaborate a bit (what kind of problem you were solving)? And did you use JNA?

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

jslade


Total Posts: 1136
Joined: Feb 2007
 
Posted: 2018-04-24 23:32
No I used JNI (JNA was total shit at the time).

"Learning, n. The kind of ignorance distinguishing the studious."

Maggette


Total Posts: 1056
Joined: Jun 2007
 
Posted: 2018-04-25 07:49
Thx

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...
Previous Thread :: Next Thread 
Page 1 of 1