most recent 30 from http://mathoverflow.net 2013-05-25T01:39:30Z http://mathoverflow.net/feeds/question/41994 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language Tim van Beek 2010-10-13T08:46:37Z 2012-07-17T09:16:34Z

<p>I'm looking for a software library with a scope similar to "numerical recipes", but implemented in a modern programming language. "Modern" in this context means to me: object oriented (not C or Fortran), running in a virtual machine with garbage collection (not C++), and with support for functional programming. Examples therefore would be: F#, C# from the .NET framework, Scala and to a lesser extend Java (no functional programming yet).</p> <p>"Scope" means that it should cover all algorithms that are taught in, say, a general, one or two year(s) long introduction to numerical analysis, starting with linear algebra, up to partial differential equations and stochastic processes. </p> <p>In addition I am interested if any research group using high performance computing uses a language that fits the description, and if not, why not. It is "common knowledge" in the software industry that one should no longer worry about performance problems of languages running in a virtual machine compared to languages compiled directly to machine code, e.g. Java versus C++. Any experience from high performance computing that affirms or contradicts this would be interesting to me, too.</p> <p>Addendum after reading the first three answers (rather than commenting them individually): My motivation for asking about these "modern" programming language is this: It takes some effort to learn a programming language, much commercial software is written in Java, often not because Java is designed for the kind of problem people have to solve, but because there are many programmers out there that know Java, there is a big open source community, and, finally, "nobody gets fired for choosing Java" (paraphrasing "nobody gets fired for buying IBM"). This outweighs the fact that Java is often clearly not the best choice from a pure language point of view. A software library in a "modern" programming language could attract more people (maybe more contributors if it is open source), for this reason, than a library in a much better suited, but less known, programming language. </p> <p>Edit: To a certain extend the Sage project and software is what I was looking for, although it employs Python as a higher level language which I don't think is a good choice. (http://www.sagemath.org/)</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/41997#41997 Alex Bartel 2010-10-13T09:18:23Z 2010-10-13T09:18:23Z

<p>I think, your idea of "modern" is not very practical, when it comes to computational mathematics. To people, who do research on algorithms, the one crucial measurement of their success is running time. This is also very important to most mathematicians, because hypotheses are tested for very large numbers/objects. Your "modern" languages are mainly geared at speeding up the design and coding process and not the running time of the programs. Implementing mathematical algorithms in C# would have more of an educational purpose than a practical one, and if someone tries to sell you such a thing, you shouldn't buy it.</p> <p>I don't know much about the "common knowledge" that you are referring to, or in fact about the software industry, but fact is that when people write computer algebra packages, they still sometimes implement the crucial and most used parts in Assembler! Your comparison "Java versus C++" misses the point. Already C++ produces so much more overhead than C, say, that it would rarely be the language of choice for computer algebra packages, where efficiency is crucial. Don't get me wrong, I recognise the huge advantages of object oriented programming languages, but they are at their best if they are used for the things they were design for.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/42000#42000 Neel Krishnaswami 2010-10-13T09:35:43Z 2010-10-13T09:35:43Z

<p>This is still something of an open problem. The reason is that achieving high performance on numerical codes takes fairly close attention to things like data locality and the details of the cache hierarchy -- failing to do so can easily cost a factor of 10 or 20 in performance. So compiling high-performance numerical code from high-level programs requires fairly sophisticated language and compiler technology. </p> <ul> <li><p>The state of the art at the moment for high-level numerical programming is probably something like <a href="http://www.sac-home.org/" rel="nofollow">SAC</a>, which is a purely functional language which uses ideas from functional programming, linear types, and nested data parallelism to efficiently parallelize and SIMD-ize high level specifications of array processing programs. This builds on ideas from older research languages like Sisal and NESL. </p></li> <li><p>Another example is the <a href="http://www.cs.bu.edu/~hwxi/ATS/ATS.html" rel="nofollow">ATS project</a>, which is a functional language which uses dependent type theory and linear types to support generating very efficient code. I don't think they have explicitly focused on numerical codes, though they have investigated high-performance systems programming more generally.</p></li> <li><p>Another commonly-used approach is to write a high-level program which generates the low-level code, rather than writing it by hand. This is used by the <a href="http://www.fftw.org/" rel="nofollow">FFTW package</a> ("Fastest Fourier Transform in the West"), which is an Ocaml program (written in a very higher-order functional style) which generates the C code in the FFTW package. </p></li> </ul>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/42001#42001 Federico Poloni 2010-10-13T09:39:44Z 2010-10-13T09:39:44Z

<p>Try <a href="http://numpy.scipy.org/" rel="nofollow">http://numpy.scipy.org/</a> (assuming you're fine with an interpreted language -- otherwise, there is a python compiler, but I know very little about it).</p> <p>Why languages like Java are not widespread? Well, my view on the subject is the following. You have two kinds of people who do numerical mathematics:</p> <ul> <li>Some need to squeeze every instruction of performance out of their mission-critical software, and naturally they are worried about aliasing, garbage processing, cache misses. Thus they use plain C or even Fortran, and use all the voodoo they know to make it run faster.</li> <li>Some don't, as they develop algorithms for testing/research only, or where the sheer computing time is not the bottleneck. They generally use Matlab, since it is much easier to work with, and as long as you avoid (or recode in C) tight <code>for</code> loops has a reasonable performance.</li> </ul>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/47785#47785 tristes_tigres 2010-11-30T11:40:57Z 2010-11-30T12:08:12Z

<p>A number of numerical libraries in Java can be found at <a href="http://math.nist.gov/javanumerics/" rel="nofollow">JavaNumerics</a></p> <blockquote> <p>"Modern" in this context means to me: object oriented (not C or Fortran)</p> </blockquote> <p>Fortran has object-oriented features since 2003, see, for instance <a href="ftp://ftp.nag.co.uk/sc22wg5/N1551-N1600/N1579.pdf" rel="nofollow">Fortran working group note 5</a> (IMHO object-orientation has been oversold, in general, and does not belong in Fortran, specifically, but the Fortran standards committee didn't ask me, alas)</p> <p>Commercial libraries that have been ported to Fortran 2003 include IMSL and NAG. Opensource library <a href="http://netlib.org/lapack/index.html" rel="nofollow">LAPACK</a> is F90, but if you are going to do numerical work, chances are, you'll <strong>have</strong> to use it.</p> <blockquote> <p>A software library in a "modern" programming language could attract more people (maybe more contributors if it is open source), for this reason, than a library in a much better suited, but less known, programming language. </p> </blockquote> <p>It is much harder to write robust numerical code, than to contribute to an average opensource project. I believe, that the choice of language is a tertiary concern, - understanding the mathematical subject matter and error anlysis takes more work than learning another (procedural) programming language.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/79795#79795 Jeny 2011-11-02T02:18:16Z 2011-11-02T02:18:16Z

<p>I think <a href="http://jwork.org/jhepwork" rel="nofollow">jHepWork</a> is very attractive and modern math software (written in Java and uses Jython as a macro language)</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/79814#79814 joro 2011-11-02T09:43:47Z 2011-11-02T09:43:47Z

<p>An advantage of sage is that it wraps high performance libraries (asm/C/C++) in python.</p> <p>I don't mean to start a language flamewar, but when I run some computation for more than a day, I definitely pick underlying C/C++ or something professionally coded in assembler and I suppose this is common practice.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/101334#101334 David Ketcheson 2012-07-04T18:43:37Z 2012-07-04T18:43:37Z

<p>There are several good answers already, but none of them so far deal with <strong>high performance computing</strong>, which nowadays means computing on tens or hundreds of thousands of cores. I am aware of only three Python codes that have scaled to this level:</p> <ul> <li><a href="http://numerics.kaust.edu.sa/pyclaw/" rel="nofollow">PyClaw</a> (my code -- you can read about it and my take on HPC software development in Python in <a href="http://arxiv.org/abs/1111.6583" rel="nofollow">this paper</a>)</li> <li><a href="https://wiki.fysik.dtu.dk/gpaw/" rel="nofollow">GPAW</a></li> <li><a href="http://www.ctcms.nist.gov/fipy/" rel="nofollow">FiPy</a></li> </ul> <p>Of course, you can't write a performant numerical code entirely in Python, so typically these use a compiled language for performance-critical parts (in PyClaw, we use both Fortran and C). One reason for choosing Python is that automated tools like f2py and Cython make it easy to incorporate compiled code.</p> <p>I do not believe there are any high performance computing codes written in the languages you mention. I suspect modern supercomputers don't support those languages.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/102084#102084 PaPiro 2012-07-13T00:30:56Z 2012-07-17T09:16:34Z

<p>I recommend the Enthought Python Distribution, available <a href="http://www.enthought.com/products/epd.php" rel="nofollow">here</a>. From the EPD homepage:</p> <p><em>The Enthought Python Distribution provides scientists with a comprehensive set of tools to perform rigorous data analysis and visualization. Python, distinguished by its flexibility, coherence, and ease-of-use, is rapidly becoming the programming language of choice for researchers worldwide. EPD extends this capacity with a powerful collection of Python libraries to enable interactive technical computing and cross-platform rapid application development.</em></p> <p>There are free academic versions (Windows, Mac OS, Linux), very easy to install.</p> <p>DISCLAIMER: I am not affiliated directly or indirectly with Enthought.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/102088#102088 tergi 2012-07-13T01:10:48Z 2012-07-13T01:10:48Z

<p>You should use code from Numerical Recipes. If you do this, then when people ask you for your code you can tell them that you aren't allowed to give it to them because it would violate the licence. Then people will stop bugging you and you won't have to reveal how messy your code is.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/102090#102090 kid551 2012-07-13T01:48:40Z 2012-07-13T01:48:40Z

<p>In my opinion, high performance computing focuses more on the efficiency of computing, especially in the area of scientific computing. So the C with high performance is undisputed a good choice.</p> <p>Moreover, in parallel computing which is a branch of high performance computing, whose enviroment is Linux terminal, programmers should deal with much more details of the data transfering and allocation by using MPI library which is more convenient implemented by C or Fortran.</p> <p>From my view point, especially in parallel programming, high performance computing is still at the status of developing and exploring. And a lot of details need to be dealt with including the standard, mechanism,abstract model of the parallel programming.(I think maybe the MPI is not the appropriate approach to describing parallel programming ultimately but is a good way nowadays.) </p> <p>When people make them clear, I think the modern programming languages as you described will be adopted generally in high performance computing.</p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/102132#102132 robot 2012-07-13T11:17:05Z 2012-07-17T08:23:36Z

<p>It is not entirely clear to me what are you looking for and why. </p> <p>If you want the best performance you are almost surely bound to use compiled language such as Fortran<a href="http://www.scribd.com/doc/99011784/Metcalf-Modern-Fortran-Explained-7th" rel="nofollow">1</a>, C++ or C. Of course, you can always use almost any "higher level language" such as Python, Ruby or whatnot to glue together routines from libraries written in some "low level language" as C, Fortran, etc. Octave, Matlab and Sage come to mind. NumPy is quite good example of this approach, since almost all of its core functions are written in C (i.e. LAPACK). </p> <p>If, on the other hand, you want to experiment with algorithms themselves, you can implement them in Haskell or (oca)ML or some other "mathematicians friendly" language. Also, succinct syntax and lack of side effects means it's much easier to prove correctness. Moreover there are area specific systems/languages as LiE, Macaulay2, Singular, GAP, ...</p> <p>Writing basic routines (such as those from Numerical Recipes) in any other language than C, C++ or Fortran means that</p> <ul> <li>you are doing an exercise </li> <li>you are trying to solve a problem in a language where you can't use functions from libraries written in C/C++/Fortran</li> <li>you need numerical routines in a big (i.e. not feasible to rewrite in C/C++/Fortran) project where calling an external library causes unwanted overhead (this can be the case for example with Java).</li> </ul> <p><a href="http://www.scribd.com/doc/99011784/Metcalf-Modern-Fortran-Explained-7th" rel="nofollow">1</a> Please note that modern Fortran is quite high level language with functional and object-oriented features.</p> <p>edit:</p> <p>I am not a Fortran programmer myself, so what follows are information relayed from a friend of mine (physicist and active Fortran programmer). I realize that this is not what you are asking for, but it may be helpful nevertheless. </p> <p>books: </p> <ul> <li>Metcalf, Reid, Cohen - <a href="http://www.scribd.com/doc/99011784/Metcalf-Modern-Fortran-Explained-7th" rel="nofollow">Modern Fortran Explained</a></li> <li>Chapman - Fortran 95/2003 for Scientists &amp; Engineers</li> </ul> <p>When starting, it's best to learn Fortran 95/90 first. For a gentle introduction see <a href="http://www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html" rel="nofollow">http://www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html</a></p> <p>For an overview of the language see <a href="http://en.wikipedia.org/wiki/Fortran_language_features" rel="nofollow">http://en.wikipedia.org/wiki/Fortran_language_features</a> (mostly written by Michael Metcalf; see the book above).</p> <p>As for OOP, it was possible in Fortran 95 using various hacks, but another revision of the language (Fortran 2003) included OOP features into the language norm itself. For technical account on Fortran 2003 features see <a href="ftp://ftp.nag.co.uk/sc22wg5/N1551-N1600/N1579.pdf" rel="nofollow">ftp://ftp.nag.co.uk/sc22wg5/N1551-N1600/N1579.pdf</a> For tutorial on OOP in Fortran 2003 see <a href="http://www.pgroup.com/lit/articles/insider/v3n1a3.htm" rel="nofollow">http://www.pgroup.com/lit/articles/insider/v3n1a3.htm</a> and <a href="http://www.pgroup.com/lit/articles/insider/v3n2a2.htm" rel="nofollow">http://www.pgroup.com/lit/articles/insider/v3n2a2.htm</a></p> <p>Fortran is still evolving, the latest standard being Fortran 2008 which introduced concurrent programming techniques:<br> <a href="http://www.training.prace-ri.eu/uploads/tx_pracetmo/coarrayvideo1.pdf" rel="nofollow">http://www.training.prace-ri.eu/uploads/tx_pracetmo/coarrayvideo1.pdf</a> <br> <a href="http://www.training.prace-ri.eu/uploads/tx_pracetmo/coarrayvideo2.pdf" rel="nofollow">http://www.training.prace-ri.eu/uploads/tx_pracetmo/coarrayvideo2.pdf</a> <br> <a href="http://www.training.prace-ri.eu/uploads/tx_pracetmo/coarrayvideo3.pdf" rel="nofollow">http://www.training.prace-ri.eu/uploads/tx_pracetmo/coarrayvideo3.pdf</a></p> <p>I should add that there probably isn't a compiler that supports all the features of the standard (2003). </p>

http://mathoverflow.net/questions/41994/basic-software-libraries-for-numerical-analysis-using-modern-programming-language/102199#102199 Alasdair McAndrew 2012-07-14T01:47:09Z 2012-07-14T01:47:09Z

<p>You might also check out the GNU Scientific Library (GSL) at <a href="http://www.gnu.org/software/gsl/" rel="nofollow">http://www.gnu.org/software/gsl/</a>, for which there are bindings and wrappers for other languages. Or you could forgo OOP and write your programs in C, which does at least have the advantage of being extremely fast and efficient. And I believe you can compile GSL so it is optimized for your system. I believe there is (or was) a Haskell wrapper for GSL, but I can't find it.</p> <p>Most high level systems (Matlab, Octave, Sage etc) allow you to write routines in C and link them in, thus getting the best of both worlds: the speed of C and the ease of use of a higher level computing environment. I guess it all comes down to what sort of algorithms you are using: big slow ones (such as Monte Carlo routines with 10^20 random numbers) or something quick and straightforward, such as quadrature.</p> <p>Over the years I've used Maple, Matlab, MuPAD, Sage, Octave, C, Java, and others, and I've written (or at least blogged) about them all. I can't prescribe one over the other - you have to decide for yourself which best suits your usage, expertise, and requirements.</p>