Discussion:
Parsing Pharo syntax to C/C++
(too old to reply)
kilon alios
2014-09-15 09:04:23 UTC
Permalink
Is there a way to convert code from pharo to c or c++ ? Does pettit parser
or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/ae00f96d/attachment.html>
Santiago Bragagnolo
2014-09-15 09:28:24 UTC
Permalink
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.

Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit parser
or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/6c84fbe5/attachment.html>
Serge Stinckwich
2014-09-15 09:33:14 UTC
Permalink
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Both are subset of Smalltalk. This is best path to follow I guess:
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.

Regards,
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
phil
2014-09-15 10:31:02 UTC
Permalink
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich <
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the result.

Phil
Post by Serge Stinckwich
Regards,
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/b3b2ccdf/attachment.html>
Serge Stinckwich
2014-09-15 11:28:42 UTC
Permalink
Post by phil
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the result.
Yes, interesting idea, instead of generating strings!
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
kilon alios
2014-09-15 11:39:19 UTC
Permalink
WOW this CCodeGenerator is great , I have downloaded it and tried the
example and it generated the 'generated.c" file . Awesome !!!! thank you
all

Ok how about this visitor thing ? any links to it , no idea what that is
and how to use it in pharo. Is it a way to code like continuations ?

About LLVM sound very cool and I was googling about that few hours ago but
from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)

So it looks like I will be sticking with Pharo after all :D


On Mon, Sep 15, 2014 at 2:28 PM, Serge Stinckwich <
Post by Santiago Bragagnolo
Post by phil
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project
related
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the
result.
Yes, interesting idea, instead of generating strings!
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/bc193b43/attachment.html>
phil
2014-09-15 12:02:21 UTC
Permalink
Post by kilon alios
WOW this CCodeGenerator is great , I have downloaded it and tried the
example and it generated the 'generated.c" file . Awesome !!!! thank you
all
Ah ah, happy to have made you day. Yes, Slang is the awesome thing for
integrating things. You can debug in Pharo, and then compile to C. How cool
is that?
Post by kilon alios
Ok how about this visitor thing ? any links to it , no idea what that is
and how to use it in pharo. Is it a way to code like continuations ?
Check the FileSystem-Core-Implementation package.

Look at the FileSystemGuide and how ti works, including the
FileSystemVisitor and its Collect and Select visitors down there.

It is interesting material to get to terms with the Visitor/Guide thing.

I reused/cloned a ton of this code for my current project where I do have
to navigate network equipement structures and generate probes along the way.

A tad mind twisting at the beginning but very useful once you get it.

Once you have the guide, you can visit all the way you want. Like here, I
generate HTML tree controls, D3 graphics, SNMP probes etc.

There are samples also for AST, but this is a tad too much for me at the
moment.
Post by kilon alios
About LLVM sound very cool and I was googling about that few hours ago but
from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)
So it looks like I will be sticking with Pharo after all :D
Ah, swallowing the red pill is nearing.

Enjoy,
Phil
Post by kilon alios
On Mon, Sep 15, 2014 at 2:28 PM, Serge Stinckwich <
Post by Santiago Bragagnolo
Post by phil
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project
related
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the
result.
Yes, interesting idea, instead of generating strings!
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/10e97b68/attachment-0001.html>
Tudor Girba
2014-09-15 12:11:11 UTC
Permalink
Please pay careful attention: this is one of those Pharo libraries that
might not meet industry standard. But, it is so small and elegant that even
an amateur can have a say with it :).

The interesting thing about the red pill is that the reality that comes
after taking the pill is less clean than the one from before the pill, But,
it is richer!

You can still choose the world you want to live in :)

Doru


On Mon, Sep 15, 2014 at 2:02 PM, phil at highoctane.be <phil at highoctane.be>
Post by phil
Post by kilon alios
WOW this CCodeGenerator is great , I have downloaded it and tried the
example and it generated the 'generated.c" file . Awesome !!!! thank you
all
Ah ah, happy to have made you day. Yes, Slang is the awesome thing for
integrating things. You can debug in Pharo, and then compile to C. How cool
is that?
Post by kilon alios
Ok how about this visitor thing ? any links to it , no idea what that is
and how to use it in pharo. Is it a way to code like continuations ?
Check the FileSystem-Core-Implementation package.
Look at the FileSystemGuide and how ti works, including the
FileSystemVisitor and its Collect and Select visitors down there.
It is interesting material to get to terms with the Visitor/Guide thing.
I reused/cloned a ton of this code for my current project where I do have
to navigate network equipement structures and generate probes along the way.
A tad mind twisting at the beginning but very useful once you get it.
Once you have the guide, you can visit all the way you want. Like here, I
generate HTML tree controls, D3 graphics, SNMP probes etc.
There are samples also for AST, but this is a tad too much for me at the
moment.
Post by kilon alios
About LLVM sound very cool and I was googling about that few hours ago
but from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)
So it looks like I will be sticking with Pharo after all :D
Ah, swallowing the red pill is nearing.
Enjoy,
Phil
Post by kilon alios
On Mon, Sep 15, 2014 at 2:28 PM, Serge Stinckwich <
Post by Santiago Bragagnolo
Post by phil
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang.
Is
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project
related
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the
result.
Yes, interesting idea, instead of generating strings!
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
--
www.tudorgirba.com

"Every thing has its own flow"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/8f8c69e0/attachment.html>
phil
2014-09-15 13:06:56 UTC
Permalink
Post by Tudor Girba
Please pay careful attention: this is one of those Pharo libraries that
might not meet industry standard. But, it is so small and elegant that even
an amateur can have a say with it :).
The interesting thing about the red pill is that the reality that comes
after taking the pill is less clean than the one from before the pill, But,
it is richer!
Exactly!!!!
Post by Tudor Girba
You can still choose the world you want to live in :)
Get me my box of red pills back :-)

Phil
Post by Tudor Girba
Doru
On Mon, Sep 15, 2014 at 2:02 PM, phil at highoctane.be <phil at highoctane.be>
Post by phil
Post by kilon alios
WOW this CCodeGenerator is great , I have downloaded it and tried the
example and it generated the 'generated.c" file . Awesome !!!! thank you
all
Ah ah, happy to have made you day. Yes, Slang is the awesome thing for
integrating things. You can debug in Pharo, and then compile to C. How cool
is that?
Post by kilon alios
Ok how about this visitor thing ? any links to it , no idea what that is
and how to use it in pharo. Is it a way to code like continuations ?
Check the FileSystem-Core-Implementation package.
Look at the FileSystemGuide and how ti works, including the
FileSystemVisitor and its Collect and Select visitors down there.
It is interesting material to get to terms with the Visitor/Guide thing.
I reused/cloned a ton of this code for my current project where I do have
to navigate network equipement structures and generate probes along the way.
A tad mind twisting at the beginning but very useful once you get it.
Once you have the guide, you can visit all the way you want. Like here, I
generate HTML tree controls, D3 graphics, SNMP probes etc.
There are samples also for AST, but this is a tad too much for me at the
moment.
Post by kilon alios
About LLVM sound very cool and I was googling about that few hours ago
but from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)
So it looks like I will be sticking with Pharo after all :D
Ah, swallowing the red pill is nearing.
Enjoy,
Phil
Post by kilon alios
On Mon, Sep 15, 2014 at 2:28 PM, Serge Stinckwich <
On Mon, Sep 15, 2014 at 12:31 PM, phil at highoctane.be <
Post by phil
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang.
Is
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project
related
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the
result.
Yes, interesting idea, instead of generating strings!
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
--
www.tudorgirba.com
"Every thing has its own flow"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/7ff645cd/attachment-0001.html>
kilon alios
2014-09-15 12:21:31 UTC
Permalink
thanks I am looking at it

is this relevant to pharo ? --> http://en.wikipedia.org/wiki/Visitor_pattern

Sure if I can stay in pharo while coding in C/C++ then why not ? Even if I
still have to do work outside pharo. The problem is beginners like me do
not realize the potential of their tools because of lack of experience, so
its more likely I give up than you who you are a much more experienced
pharoer than me.

Afterall Neo needed Morpheas to show him that he is the chosen one ;)

I invested a year of my life in pharo and I would love to invest a lot more
and benefit back the community. This way everyone is a winner.

Also Thierry mentioned Ring which led me to this
http://rmod.lille.inria.fr/archives/papers/Uqui11a-RingJournalPaper-CSSJournal.pdf
from
the looks of it is a source code analyzer ?

much to study , a lot of potential and another happy Pharo user :)

On Mon, Sep 15, 2014 at 3:02 PM, phil at highoctane.be <phil at highoctane.be>
Post by phil
Post by kilon alios
WOW this CCodeGenerator is great , I have downloaded it and tried the
example and it generated the 'generated.c" file . Awesome !!!! thank you
all
Ah ah, happy to have made you day. Yes, Slang is the awesome thing for
integrating things. You can debug in Pharo, and then compile to C. How cool
is that?
Post by kilon alios
Ok how about this visitor thing ? any links to it , no idea what that is
and how to use it in pharo. Is it a way to code like continuations ?
Check the FileSystem-Core-Implementation package.
Look at the FileSystemGuide and how ti works, including the
FileSystemVisitor and its Collect and Select visitors down there.
It is interesting material to get to terms with the Visitor/Guide thing.
I reused/cloned a ton of this code for my current project where I do have
to navigate network equipement structures and generate probes along the way.
A tad mind twisting at the beginning but very useful once you get it.
Once you have the guide, you can visit all the way you want. Like here, I
generate HTML tree controls, D3 graphics, SNMP probes etc.
There are samples also for AST, but this is a tad too much for me at the
moment.
Post by kilon alios
About LLVM sound very cool and I was googling about that few hours ago
but from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)
So it looks like I will be sticking with Pharo after all :D
Ah, swallowing the red pill is nearing.
Enjoy,
Phil
Post by kilon alios
On Mon, Sep 15, 2014 at 2:28 PM, Serge Stinckwich <
Post by Santiago Bragagnolo
Post by phil
On Mon, Sep 15, 2014 at 11:33 AM, Serge Stinckwich
Post by Serge Stinckwich
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang.
Is
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project
related
Post by phil
Post by Serge Stinckwich
Post by Santiago Bragagnolo
with EToys), but i am not sure about how complete is.
define your own DSL for your needs and implement a visitor to do code
generation.
We have done that for epidemiological modeling.
Or combine both: visitor which uses CCodeGenerator for emitting the
result.
Yes, interesting idea, instead of generating strings!
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/18979107/attachment-0001.html>
Damien Cassou
2014-09-15 14:33:37 UTC
Permalink
Post by kilon alios
is this relevant to pharo ? --> http://en.wikipedia.org/wiki/Visitor_pattern
yes, this is what they are talking about!
--
Damien Cassou
http://damiencassou.seasidehosting.st

"Success is the ability to go from one failure to another without
losing enthusiasm."
Winston Churchill
Alain Rastoul
2014-09-19 23:11:37 UTC
Permalink
Post by kilon alios
thanks I am looking at it
is this relevant to pharo ? --> http://en.wikipedia.org/wiki/Visitor_pattern
Hi,
Of course, the only difference between what you will find in
java,dotNet,c++ examples and Smalltalk is that the latter does not
support method signature because of it's typeless feature and you will
have to define different methods.
The smalltalk equivalent of (java code)
void visit(Wheel wheel);
void visit(Engine engine);
void visit(Body body);
void visit(Car car);
would be the following methods:
visitWheel: aWhell
visitEngine: anEngine
visitBody: aBody
visitCar: aCar

A small disadvantage, but everything has a price ... :)

If you are interested in object patterns, you should buy the GOF book
http://en.wikipedia.org/wiki/Design_Patterns
It's a must have
and it's smalltalk companion too
http://www.amazon.fr/The-Design-Patterns-Smalltalk-Companion/dp/0201184621
see chapters here:
http://stephane.ducasse.free.fr/FreeBooks/SmalltalkDesignPatternCompanion/



Regards,

Alain
Thierry Goubier
2014-09-15 12:26:46 UTC
Permalink
Post by kilon alios
About LLVM sound very cool and I was googling about that few hours ago but
from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)
LLVM is an industrial-class open source project included in, I believe, all
GPU drivers (for compilation of CUDA and OpenCL code).

It is also one of the best documented low-level compilation toolkits with
performance to match the other open source compiler (gcc). LLVM-IR is
fairly well documented.
Post by kilon alios
So it looks like I will be sticking with Pharo after all :D
:)

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/b02a8313/attachment.html>
kilon alios
2014-09-15 12:44:27 UTC
Permalink
Yeah I think I mixed it up with something else, anyway I found a very nice
tutorial how to make a working compiler with LLVM

http://www.ibm.com/developerworks/library/os-createcompilerllvm1/index.html

http://www.ibm.com/developerworks/library/os-createcompilerllvm2/

looks like nativeboost on steroids !!!!


On Mon, Sep 15, 2014 at 3:26 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by kilon alios
About LLVM sound very cool and I was googling about that few hours ago
but from what I have read is a very undocumented part of LLVM so that maybe
easier said than done. Looks like Pharo is not the only project having
issues with documentation ;)
LLVM is an industrial-class open source project included in, I believe,
all GPU drivers (for compilation of CUDA and OpenCL code).
It is also one of the best documented low-level compilation toolkits with
performance to match the other open source compiler (gcc). LLVM-IR is
fairly well documented.
Post by kilon alios
So it looks like I will be sticking with Pharo after all :D
:)
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/5905eabb/attachment-0001.html>
Ben Coman
2014-09-15 13:47:38 UTC
Permalink
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/e9fab124/attachment.html>
phil
2014-09-15 10:29:42 UTC
Permalink
Slang has been externalized by Pavel. So, Smalltalk to C works.

Works nicely, even if there were a few glitches (like code generated twice
at one point).
Nothing unfixable, I got the beast working.

Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.

PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).

Phil







On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/d6a7c443/attachment.html>
Thierry Goubier
2014-09-15 11:28:52 UTC
Permalink
Hi Phil,

thanks for the update on Slang to C. Allways significant to have that.

Two open questions:

- would a slang to x86 asm via NativeBoost be doable / a nice target?

- would targetting LLVM-IR be of interest?

Thierry
Post by phil
Slang has been externalized by Pavel. So, Smalltalk to C works.
Works nicely, even if there were a few glitches (like code generated twice
at one point).
Nothing unfixable, I got the beast working.
Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.
PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).
Phil
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/aacf590a/attachment.html>
phil
2014-09-15 11:56:34 UTC
Permalink
On Mon, Sep 15, 2014 at 1:28 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
I have to admit that I am happy to have Slang to C and it gets out of my
competence from there :-)

Now, Slang to asm, that's quite a chasm to cross. If what asm does is doing
syscalls, well, I don't see the added value right away as NB Assembler
would do that already no?

Now, I am on Linux for about all of my Pharo code, so, C is nice enough.
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
Oh, that would be interesting. In order to get the IR interpreter and
running things all over. Now, this should be extended to the whole VM then.
But not sure we want that before we get 64 bits...

Ah, I should have taken the research career :-)

Phil
Post by Thierry Goubier
Thierry
Post by phil
Slang has been externalized by Pavel. So, Smalltalk to C works.
Works nicely, even if there were a few glitches (like code generated
twice at one point).
Nothing unfixable, I got the beast working.
Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.
PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).
Phil
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is
the pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/84c0f1fc/attachment.html>
Thierry Goubier
2014-09-15 12:33:41 UTC
Permalink
Post by phil
On Mon, Sep 15, 2014 at 1:28 PM, Thierry Goubier <
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
I have to admit that I am happy to have Slang to C and it gets out of my
competence from there :-)
Yes, I believe it is very cool as it is. You need a full C toolchain around
to make it work.
Post by phil
Now, Slang to asm, that's quite a chasm to cross. If what asm does is
doing syscalls, well, I don't see the added value right away as NB
Assembler would do that already no?
Hum, I would more like SSE and AVX code done this way: matrix
multiplications, bitmap processing, heavily used code, SciSmalltalk stuff
on very large datasets.

To see NB only used as a way to write syscalls :(:(:(
Post by phil
Now, I am on Linux for about all of my Pharo code, so, C is nice enough.
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
Oh, that would be interesting. In order to get the IR interpreter and
running things all over. Now, this should be extended to the whole VM then.
But not sure we want that before we get 64 bits...
OpenCL, SPIR code generation and link to Cog.

The ability to generate better code than when targetting C.
Post by phil
Ah, I should have taken the research career :-)
Hum, you know that researchers are allways on the lookout for SMEs to
collaborate on projects :-)

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/257c1c03/attachment.html>
phil
2014-09-15 13:13:36 UTC
Permalink
On Mon, Sep 15, 2014 at 2:33 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by phil
On Mon, Sep 15, 2014 at 1:28 PM, Thierry Goubier <
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
I have to admit that I am happy to have Slang to C and it gets out of my
competence from there :-)
Yes, I believe it is very cool as it is. You need a full C toolchain
around to make it work.
Post by phil
Now, Slang to asm, that's quite a chasm to cross. If what asm does is
doing syscalls, well, I don't see the added value right away as NB
Assembler would do that already no?
Hum, I would more like SSE and AVX code done this way: matrix
multiplications, bitmap processing, heavily used code, SciSmalltalk stuff
on very large datasets.
Indeed! I'd live to generate some CUDA code from Pharo. But the toolchain
goes C code which gets compiled through some "black-hole toolchain" in the
NVidia SDK with host and GPU blocks.

That's too much to handle for my tastes, so the best is for me to have an
engine (or for a better metaphor) and using Pharo to drive/steer/harness it.
Post by Thierry Goubier
To see NB only used as a way to write syscalls :(:(:(
You opened my eyes with the previous comment. Thx.
Post by Thierry Goubier
Post by phil
Now, I am on Linux for about all of my Pharo code, so, C is nice enough.
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
Oh, that would be interesting. In order to get the IR interpreter and
running things all over. Now, this should be extended to the whole VM then.
But not sure we want that before we get 64 bits...
OpenCL, SPIR code generation and link to Cog.
Indeed! CUDA =-)
Post by Thierry Goubier
The ability to generate better code than when targetting C.
True too. But aren't those compilers smart enough already? With current
CPUs, well, it takes a strong man to do better things than the compiler it
seems. Or am I mistaken?
Post by Thierry Goubier
Post by phil
Ah, I should have taken the research career :-)
Hum, you know that researchers are allways on the lookout for SMEs to
collaborate on projects :-)
Duly noted :-) Feel free to tickle me :-)
Post by Thierry Goubier
Thierry
Phil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/35417096/attachment.html>
Thierry Goubier
2014-09-15 13:25:25 UTC
Permalink
Post by phil
On Mon, Sep 15, 2014 at 2:33 PM, Thierry Goubier <
Post by Thierry Goubier
Hum, I would more like SSE and AVX code done this way: matrix
multiplications, bitmap processing, heavily used code, SciSmalltalk stuff
on very large datasets.
Indeed! I'd live to generate some CUDA code from Pharo. But the toolchain
goes C code which gets compiled through some "black-hole toolchain" in the
NVidia SDK with host and GPU blocks.
You may be interested to contact Bernard Pottier. His NetGen toolkit does
CUDA code generation. NetGen is VisualWorks only for the moment, even if I
know someone who would like to have it in Pharo (well, two guys at least).
Post by phil
That's too much to handle for my tastes, so the best is for me to have an
engine (or for a better metaphor) and using Pharo to drive/steer/harness it.
Yes. I intended to do things around that, but didn't found the time.
Post by phil
Indeed! CUDA =-)
OpenCL, SPIR code generation and link to Cog.
Post by Thierry Goubier
The ability to generate better code than when targetting C.
True too. But aren't those compilers smart enough already? With current
CPUs, well, it takes a strong man to do better things than the compiler it
seems. Or am I mistaken?
You are. A lot may be done if your input language is not C, but this is not
allways relevant to the Pharo situation.

I should know a lot more about LLVM-IR generation in a few months time.
I'll say if I see easy gains when doing Smalltalk code.
Post by phil
Post by Thierry Goubier
Post by phil
Ah, I should have taken the research career :-)
Hum, you know that researchers are allways on the lookout for SMEs to
collaborate on projects :-)
Duly noted :-) Feel free to tickle me :-)
Noted :)

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/dffd7d7f/attachment-0001.html>
kilon alios
2014-09-15 14:38:32 UTC
Permalink
Thanks Damien :)

so I want to further explore Slang , that means downloading the VMMaker , I
am following the post of Mariano here
https://marianopeck.wordpress.com/tag/vmmaker/

and install VMMaker like this

1
2
3
4
5
6
Deprecation raiseWarning: false.
Gofer new
squeaksource:'MetacelloRepository';
package:'ConfigurationOfVMMaker';
load.
((Smalltalk at: #ConfigurationOfVMMaker) project version: '1.5') load.


Question does CCodeGenerator contain the entire Slang ?

I am downloading VMMaker to find how Slang plays with pointers and manual
memory management so if you have any pointer (pun not intended) I will
greatly appreciate it.

On Mon, Sep 15, 2014 at 4:25 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by phil
On Mon, Sep 15, 2014 at 2:33 PM, Thierry Goubier <
Post by Thierry Goubier
Hum, I would more like SSE and AVX code done this way: matrix
multiplications, bitmap processing, heavily used code, SciSmalltalk stuff
on very large datasets.
Indeed! I'd live to generate some CUDA code from Pharo. But the toolchain
goes C code which gets compiled through some "black-hole toolchain" in the
NVidia SDK with host and GPU blocks.
You may be interested to contact Bernard Pottier. His NetGen toolkit does
CUDA code generation. NetGen is VisualWorks only for the moment, even if I
know someone who would like to have it in Pharo (well, two guys at least).
Post by phil
That's too much to handle for my tastes, so the best is for me to have an
engine (or for a better metaphor) and using Pharo to drive/steer/harness it.
Yes. I intended to do things around that, but didn't found the time.
Post by phil
Indeed! CUDA =-)
OpenCL, SPIR code generation and link to Cog.
Post by Thierry Goubier
The ability to generate better code than when targetting C.
True too. But aren't those compilers smart enough already? With current
CPUs, well, it takes a strong man to do better things than the compiler it
seems. Or am I mistaken?
You are. A lot may be done if your input language is not C, but this is
not allways relevant to the Pharo situation.
I should know a lot more about LLVM-IR generation in a few months time.
I'll say if I see easy gains when doing Smalltalk code.
Post by phil
Post by Thierry Goubier
Post by phil
Ah, I should have taken the research career :-)
Hum, you know that researchers are allways on the lookout for SMEs to
collaborate on projects :-)
Duly noted :-) Feel free to tickle me :-)
Noted :)
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/093bbb5f/attachment-0001.html>
kilon alios
2014-09-15 14:44:52 UTC
Permalink
hmm no that does not work I am getting this error (Cannot Resolve FreeType)
, any solutions ?

I also tried loading latest version same error

MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
MetacelloFetchingMCSpecLoader>>linearLoadPackageSpec:gofer: in Block:
linearLoadPackageSpec: packageSpec gofer: gofer...
MetacelloPharo30Platform(MetacelloPlatform)>>do:displaying:
MetacelloFetchingMCSpecLoader>>linearLoadPackageSpec:gofer:
MetacelloPackageSpec>>loadUsing:gofer:
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>linearLoadPackageSpecs:repositories:
in Block: [ :pkg | pkg loadUsing: self gofer: gofer ]
OrderedCollection>>do:
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>linearLoadPackageSpecs:repositories:
MetacelloFetchingMCSpecLoader>>linearLoadPackageSpecs:repositories: in
Block: [ super linearLoadPackageSpecs: packageSpecs repos...etc...
BlockClosure>>ensure:
MetacelloLoaderPolicy>>pushLoadDirective:during:
MetacelloLoaderPolicy>>pushLinearLoadDirectivesDuring:for:
MetacelloFetchingMCSpecLoader>>linearLoadPackageSpecs:repositories:
MetacelloFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>load
MetacelloMCVersionSpecLoader>>load
MetacelloMCVersion>>executeLoadFromArray:
MetacelloMCVersion>>fetchRequiredFromArray: in Block: [ :dict | ^ self
executeLoadFromArray: anArray ]
MetacelloPharo30Platform(MetacelloPlatform)>>useStackCacheDuring:defaultDictionary:
in Block: [ ^ aBlock value: dict ]
BlockClosure>>on:do:
MetacelloPharo30Platform(MetacelloPlatform)>>useStackCacheDuring:defaultDictionary:
MetacelloMCVersion>>fetchRequiredFromArray: in Block: [ ...
BlockClosure>>ensure:
MetacelloMCVersion>>fetchRequiredFromArray: in Block: [ ...
MetacelloPharo30Platform(MetacelloPlatform)>>do:displaying:
MetacelloMCVersion>>fetchRequiredFromArray:
MetacelloMCVersion>>doLoadRequiredFromArray: in Block: [ ...
BlockClosure>>ensure:
MetacelloMCVersion>>doLoadRequiredFromArray:
MetacelloMCVersion>>load
UndefinedObject>>DoIt
Post by kilon alios
Thanks Damien :)
so I want to further explore Slang , that means downloading the VMMaker ,
I am following the post of Mariano here
https://marianopeck.wordpress.com/tag/vmmaker/
and install VMMaker like this
1
2
3
4
5
6
Deprecation raiseWarning: false.
Gofer new
squeaksource:'MetacelloRepository';
package:'ConfigurationOfVMMaker';
load.
((Smalltalk at: #ConfigurationOfVMMaker) project version: '1.5') load.
Question does CCodeGenerator contain the entire Slang ?
I am downloading VMMaker to find how Slang plays with pointers and manual
memory management so if you have any pointer (pun not intended) I will
greatly appreciate it.
On Mon, Sep 15, 2014 at 4:25 PM, Thierry Goubier <
Post by Thierry Goubier
Post by phil
On Mon, Sep 15, 2014 at 2:33 PM, Thierry Goubier <
Post by Thierry Goubier
Hum, I would more like SSE and AVX code done this way: matrix
multiplications, bitmap processing, heavily used code, SciSmalltalk stuff
on very large datasets.
Indeed! I'd live to generate some CUDA code from Pharo. But the
toolchain goes C code which gets compiled through some "black-hole
toolchain" in the NVidia SDK with host and GPU blocks.
You may be interested to contact Bernard Pottier. His NetGen toolkit does
CUDA code generation. NetGen is VisualWorks only for the moment, even if I
know someone who would like to have it in Pharo (well, two guys at least).
Post by phil
That's too much to handle for my tastes, so the best is for me to have
an engine (or for a better metaphor) and using Pharo to drive/steer/harness
it.
Yes. I intended to do things around that, but didn't found the time.
Post by phil
Indeed! CUDA =-)
OpenCL, SPIR code generation and link to Cog.
Post by Thierry Goubier
The ability to generate better code than when targetting C.
True too. But aren't those compilers smart enough already? With current
CPUs, well, it takes a strong man to do better things than the compiler it
seems. Or am I mistaken?
You are. A lot may be done if your input language is not C, but this is
not allways relevant to the Pharo situation.
I should know a lot more about LLVM-IR generation in a few months time.
I'll say if I see easy gains when doing Smalltalk code.
Post by phil
Post by Thierry Goubier
Post by phil
Ah, I should have taken the research career :-)
Hum, you know that researchers are allways on the lookout for SMEs to
collaborate on projects :-)
Duly noted :-) Feel free to tickle me :-)
Noted :)
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/4d49ef46/attachment-0001.html>
Clément Bera
2014-09-15 12:39:17 UTC
Permalink
Hello,

Note that slang is a subset of smalltalk. The Slang compiler does not allow
to compile smalltalk to C. It allows to compile a smalltalk with restricted
message sends and classes to C.
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
Yes it would be interesting. However, by having a Slang to C compiler,
we're platform-independent, we can compile the C code to x86, x86_64 and
ARM quite easily (some part of the VM are already processor dependent, but
not so much). Targeting direct machine code implies evolving the Slang
compiler for each new assembly code we support. It sounds like a lot of
engineering work compared to our resources and the gain.
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
If you compile the C code with Clang instead of gcc, which starts to be
the case because of the lack of support for gcc in the latest Mac OS X, you
are already using LLVM IR because Clang uses it. As the VM use the GNU C
extensions to improve performance, I do not think that targeting directly
the LLVM IR would greatly improve performance. So it sounds like quite some
engineering work for no gain.

However, I think Ronie was interested in doing such work. If he succeeds
and reports performance improvement, then we can consider using his
compiler to compile the VM.
Post by Thierry Goubier
Thierry
Slang has been externalized by Pavel. So, Smalltalk to C works.
Post by phil
Works nicely, even if there were a few glitches (like code generated
twice at one point).
Nothing unfixable, I got the beast working.
Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.
PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).
Phil
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is
the pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/52cbf632/attachment.html>
Thierry Goubier
2014-09-15 13:01:55 UTC
Permalink
Post by Clément Bera
Hello,
Note that slang is a subset of smalltalk. The Slang compiler does not
allow to compile smalltalk to C. It allows to compile a smalltalk with
restricted message sends and classes to C.
Yes, I am aware of that. I remember that from the very beginnings of Squeak.

Wasn't Smalltalk/X the one which had a more complete version of that C
translation? I did an internship in a French company who had a Smalltalk to
C translator done for them a long time ago.
Post by Clément Bera
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
Yes it would be interesting. However, by having a Slang to C compiler,
we're platform-independent, we can compile the C code to x86, x86_64 and
ARM quite easily (some part of the VM are already processor dependent, but
not so much). Targeting direct machine code implies evolving the Slang
compiler for each new assembly code we support. It sounds like a lot of
engineering work compared to our resources and the gain.
It would allow JIT-type compilation experiments than a Slang-to-C chain
isn't designed for :) With a lot more work doing the various NB ports, of
course.
Post by Clément Bera
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
If you compile the C code with Clang instead of gcc, which starts to be
the case because of the lack of support for gcc in the latest Mac OS X, you
are already using LLVM IR because Clang uses it. As the VM use the GNU C
extensions to improve performance, I do not think that targeting directly
the LLVM IR would greatly improve performance. So it sounds like quite some
engineering work for no gain.
I would not suggest replacing C by LLVM-IR for VM work, in part because
LLVM-IR is not what I would call a readable source code format... But I do
know that even when doing C to C rewritting for embedded compilation, there
is some low-level code that you can't write in C.
Post by Clément Bera
However, I think Ronie was interested in doing such work. If he succeeds
and reports performance improvement, then we can consider using his
compiler to compile the VM.
Keep us posted!

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/63790aad/attachment.html>
Jan Vrany
2014-09-15 13:20:19 UTC
Permalink
Post by Thierry Goubier
Wasn't Smalltalk/X the one which had a more complete version of
that C translation?
Yes. Smalltalk/X **still** compiles Smalltalk to C (bytecode
interpreting and JITing modes are also supported, of course).

Best, Jan
Eliot Miranda
2014-09-15 16:23:33 UTC
Permalink
Hi All,

On Mon, Sep 15, 2014 at 6:01 AM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by Clément Bera
Hello,
Note that slang is a subset of smalltalk. The Slang compiler does not
allow to compile smalltalk to C. It allows to compile a smalltalk with
restricted message sends and classes to C.
Yes, I am aware of that. I remember that from the very beginnings of Squeak.
Wasn't Smalltalk/X the one which had a more complete version of that C
translation? I did an internship in a French company who had a Smalltalk to
C translator done for them a long time ago.
Post by Clément Bera
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
Yes it would be interesting. However, by having a Slang to C compiler,
we're platform-independent, we can compile the C code to x86, x86_64 and
ARM quite easily (some part of the VM are already processor dependent, but
not so much). Targeting direct machine code implies evolving the Slang
compiler for each new assembly code we support. It sounds like a lot of
engineering work compared to our resources and the gain.
It would allow JIT-type compilation experiments than a Slang-to-C chain
isn't designed for :) With a lot more work doing the various NB ports, of
course.
Post by Clément Bera
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
If you compile the C code with Clang instead of gcc, which starts to be
the case because of the lack of support for gcc in the latest Mac OS X, you
are already using LLVM IR because Clang uses it. As the VM use the GNU C
extensions to improve performance, I do not think that targeting directly
the LLVM IR would greatly improve performance. So it sounds like quite some
engineering work for no gain.
I would not suggest replacing C by LLVM-IR for VM work, in part because
LLVM-IR is not what I would call a readable source code format... But I do
know that even when doing C to C rewritting for embedded compilation, there
is some low-level code that you can't write in C.
I find this whole discussion depressing. It seems people would rather put
their energy in chasing quick fixes or other technologies instead of
contributing to the work that is being done in the existing VM. People
discuss using LLVM as if the code generation capabilities inside Cog were
somehow poor or have no chance of competing. Spur is around twice as fast
as the current memory manager, has much better support for the FFI.
Cl?ment and I, now with help from Ronie, are making excellent progress
towards an adaptive optimizer/speculative inliner that will give us similar
performance to V8 (the Google JavaScript VM, lead by Lars Bak, who
implemented the HotSpot VM (Smalltalk and Java)) et al. We are trying to
get person-power for a high-quality FFI and have a prototype for a
non-blocking VM. When we succeed C won't be any better and so it won't be
an interesting target. One will be able to program entirely in Smalltalk
and get excellent performance. But we need effort. Collaboration.

Personally I feel so discouraged when people talk about using LLVM or
libffi or whatever instead of having the courage and energy to make our
system world-class. I have the confidence in our abilities to compete with
the best and am saddened that people in the community don't value the
technology we already have and can't show faith in our abilities to improve
it further. Show some confidence and express support and above all get
involved.

Collaborators <http://www.mirandabanda.org/cogblog/collaborators/>
Cog Projects <http://www.mirandabanda.org/cogblog/cog-projects/>
Spur 1/3

Spur, a new object representa...
<http://www.slideshare.net/esug/spur-a-new-object-representation-for-cog>
Sista: Improving Cog's JIT performance 1/2

Sista: Improving Cog?s JIT pe
<http://www.slideshare.net/esug/sista-talkesug2>..
Lowcode: Redoing NativeBoost ...
<http://www.slideshare.net/esug/03-lowcodeslides>


However, I think Ronie was interested in doing such work. If he succeeds
Post by Thierry Goubier
Post by Clément Bera
and reports performance improvement, then we can consider using his
compiler to compile the VM.
Keep us posted!
Thierry
--
in hope,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/2d8cbf92/attachment.html>
phil
2014-09-15 16:30:50 UTC
Permalink
On Mon, Sep 15, 2014 at 6:23 PM, Eliot Miranda <eliot.miranda at gmail.com>
Post by Eliot Miranda
Hi All,
On Mon, Sep 15, 2014 at 6:01 AM, Thierry Goubier <
Post by Thierry Goubier
Post by Clément Bera
Hello,
Note that slang is a subset of smalltalk. The Slang compiler does not
allow to compile smalltalk to C. It allows to compile a smalltalk with
restricted message sends and classes to C.
Yes, I am aware of that. I remember that from the very beginnings of Squeak.
Wasn't Smalltalk/X the one which had a more complete version of that C
translation? I did an internship in a French company who had a Smalltalk to
C translator done for them a long time ago.
Post by Clément Bera
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
Yes it would be interesting. However, by having a Slang to C compiler,
we're platform-independent, we can compile the C code to x86, x86_64 and
ARM quite easily (some part of the VM are already processor dependent, but
not so much). Targeting direct machine code implies evolving the Slang
compiler for each new assembly code we support. It sounds like a lot of
engineering work compared to our resources and the gain.
It would allow JIT-type compilation experiments than a Slang-to-C chain
isn't designed for :) With a lot more work doing the various NB ports, of
course.
Post by Clément Bera
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
If you compile the C code with Clang instead of gcc, which starts to be
the case because of the lack of support for gcc in the latest Mac OS X, you
are already using LLVM IR because Clang uses it. As the VM use the GNU C
extensions to improve performance, I do not think that targeting directly
the LLVM IR would greatly improve performance. So it sounds like quite some
engineering work for no gain.
I would not suggest replacing C by LLVM-IR for VM work, in part because
LLVM-IR is not what I would call a readable source code format... But I do
know that even when doing C to C rewritting for embedded compilation, there
is some low-level code that you can't write in C.
I find this whole discussion depressing. It seems people would rather put
their energy in chasing quick fixes or other technologies instead of
contributing to the work that is being done in the existing VM.
Why so? I am all in for using the VM based technology. Slang is great. Now,
we need to interface with the outside world, and then it is C-based at one
point.
Post by Eliot Miranda
People discuss using LLVM as if the code generation capabilities inside
Cog were somehow poor or have no chance of competing. Spur is around twice
as fast as the current memory manager, has much better support for the FFI.
Cl?ment and I, now with help from Ronie, are making excellent progress
towards an adaptive optimizer/speculative inliner that will give us similar
performance to V8 (the Google JavaScript VM, lead by Lars Bak, who
implemented the HotSpot VM (Smalltalk and Java)) et al.
Super, that's why I bet my company business on Pharo for software
development. I trust you guys.
Post by Eliot Miranda
We are trying to get person-power for a high-quality FFI and have a
prototype for a non-blocking VM. When we succeed C won't be any better and
so it won't be an interesting target. One will be able to program entirely
in Smalltalk and get excellent performance. But we need effort.
Collaboration.
Ah, non blocking VM, a super cool thing to have. I am going through hoops
at the moment to avoid doing direct SNMP calls to devices from Pharo...
Post by Eliot Miranda
Personally I feel so discouraged when people talk about using LLVM or
libffi or whatever instead of having the courage and energy to make our
system world-class. I have the confidence in our abilities to compete with
the best and am saddened that people in the community don't value the
technology we already have and can't show faith in our abilities to improve
it further. Show some confidence and express support and above all get
involved.
That's well said. Let's race to the top!
Post by Eliot Miranda
Collaborators <http://www.mirandabanda.org/cogblog/collaborators/>
Cog Projects <http://www.mirandabanda.org/cogblog/cog-projects/>
Spur 1/3
http://youtu.be/k0nBNS1aHZ4
Spur, a new object representa...
<http://www.slideshare.net/esug/spur-a-new-object-representation-for-cog>
Sista: Improving Cog's JIT performance 1/2
http://youtu.be/X4E_FoLysJg
Sista: Improving Cog?s JIT pe
<http://www.slideshare.net/esug/sista-talkesug2>..
Lowcode: Redoing NativeBoost ...
<http://www.slideshare.net/esug/03-lowcodeslides>
All very interesting things indeed. Now, doing application level code at
the moment, I can't really dig into these. But I can use them and kick ass!
Post by Eliot Miranda
However, I think Ronie was interested in doing such work. If he succeeds
Post by Thierry Goubier
Post by Clément Bera
and reports performance improvement, then we can consider using his
compiler to compile the VM.
Keep us posted!
Thierry
--
in hope,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/5e13acb7/attachment-0001.html>
Thierry Goubier
2014-09-15 19:08:12 UTC
Permalink
Hi Eliot,
Post by Eliot Miranda
Hi All,
On Mon, Sep 15, 2014 at 6:01 AM, Thierry Goubier
I would not suggest replacing C by LLVM-IR for VM work, in part
because LLVM-IR is not what I would call a readable source code
format... But I do know that even when doing C to C rewritting for
embedded compilation, there is some low-level code that you can't
write in C.
I find this whole discussion depressing. It seems people would rather
put their energy in chasing quick fixes or other technologies instead
of contributing to the work that is being done in the existing VM.
People discuss using LLVM as if the code generation capabilities
inside Cog were somehow poor or have no chance of competing. Spur is
around twice as fast as the current memory manager, has much better
support for the FFI. Cl?ment and I, now with help from Ronie, are
making excellent progress towards an adaptive optimizer/speculative
inliner that will give us similar performance to V8 (the Google
JavaScript VM, lead by Lars Bak, who implemented the HotSpot VM
(Smalltalk and Java)) et al. We are trying to get person-power for a
high-quality FFI and have a prototype for a non-blocking VM. When we
succeed C won't be any better and so it won't be an interesting
target. One will be able to program entirely in Smalltalk and get
excellent performance. But we need effort. Collaboration.
I certainly appreciate how much we own to your work on Cog. I wouldn't
have come back to Squeak/Pharo without it (not after going through the
Squeak VM when Ian Piumarta was taking care of it), and I'm thrilled by
the performance goals you're targetting. This is great!

At the same time, I do use that C/LLVM today for automatic
vectorization, GPU support, automatic parallelisation for CPUs with over
a thousand cores, compiling for exotic embedded platforms... That work I
have to do for R (or Python) I'd also like to explore it with Pharo and
for that, FFI or NativeBoost have their uses.

Including the fact that I can run experiments proving or disproving
certain techniques/attempts without bothering Cog development until it
is proven it is worth the inclusion :)
Post by Eliot Miranda
Personally I feel so discouraged when people talk about using LLVM or
libffi or whatever instead of having the courage and energy to make
our system world-class. I have the confidence in our abilities to
compete with the best and am saddened that people in the community
don't value the technology we already have and can't show faith in our
abilities to improve it further. Show some confidence and express
support and above all get involved.
You have a point :) VM development is hard, and is vital for Squeak/Pharo.
Post by Eliot Miranda
Collaborators <http://www.mirandabanda.org/cogblog/collaborators/>
Cog Projects <http://www.mirandabanda.org/cogblog/cog-projects/>
Spur 1/3
http://youtu.be/k0nBNS1aHZ4
Spur, a new object representa...
<http://www.slideshare.net/esug/spur-a-new-object-representation-for-cog>
Sista: Improving Cog's JIT performance 1/2
http://youtu.be/X4E_FoLysJg
Sista: Improving Cog?s JIT pe
<http://www.slideshare.net/esug/sista-talkesug2>..
Lowcode: Redoing NativeBoost ...
<http://www.slideshare.net/esug/03-lowcodeslides>
You didn't announce the last one or I missed it?

Regards,

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/a0853226/attachment.html>
Ronie Salgado
2014-09-15 21:37:08 UTC
Permalink
Hello,

I am segmenting this mail in several sections.

---------------------------------------------------------------
- On Lowcode and Cog

I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.

Lowcode is currently a spec of new bytecode instructions. These
instructions can be used for:
- Implementing a C like language compiler.
- Making FFI calls

I am implementing these instructions using a feature of the new bytecode
set for SistaV1, which is called "inline primitives". Because of this,
these new instructions can be mixed freely with the standard VM bytecode
set. This also allows the Sista adaptive optimizer to inline FFI calls.

These instructions provides features for:
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.

Current I have implemented the following backends:
- A C interpreter plugin.
- A LLVM based backend.

Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.

During this work I learned a lot about Cog. Specially that Cog is missing a
better Slang generator, that allows to force better inlining and more code
reviews. There is a lot of code duplication in Cog, that can be attributed
to limitations of Slang. In my opinion, if we could use Slang not only for
building the VM we should end with a better code generator. In addition we,
need more people working in Cog. We need people that performs code reviews
and documentation of Cog.

After these weeks, I learned that working in Cogit it is not that hard. Our
biggest problem is lack of documentation. Our second problem could be the
lack of documentation about Slang.

---------------------------------------------------------------
- Smalltalk -> LLVM ?

As for having a Smalltalk -> LLVM code generator. The truth is that we will
not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.

To optimize a message send, you have to determine which is the method that
is going to respond to the message. Then you have to inline the method. And
then you can start performing the actual optimizations, such as constant
folding, common subexpressions, dead branch elimination, loop unrolling,
and a long etc.

Because we don't have information in the actual language (e.g. static types
a la C/C++/Java/C#) that tells us what is going to be the actual method
invoked by a message send, we have the following alternatives to determine
it:
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.

In other words, our best bet is in the work of Cl?ment in Sista. The only
problem with this bet are real time applications.

Real time applications requires an upper bound guarantee in their response
time. In some cases, the lack of this guarantee can be just an annoyance,
as happens in video games. In some mission critical applications the
results can not be good, if this time constraint is not met. An example of
a mission critical system could the flight controls of an airplane, or the
cooling system of a nuclear reactor.

For these application, it is not possible to rely in an adaptive optimizer
that can be triggered sometimes. In these application you have to either:
- Extend the language to hand optimize some performance critical sections
of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.

And of course, you have to perform lot of profiling.

Greetings,
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/5ceb3bcb/attachment.html>
Eliot Miranda
2014-09-15 23:46:57 UTC
Permalink
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
remember to send me code for integration. I'm eagerly waiting to use your
code!

Lowcode is currently a spec of new bytecode instructions. These
Post by Clément Bera
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new bytecode
set for SistaV1, which is called "inline primitives". Because of this,
these new instructions can be mixed freely with the standard VM bytecode
set. This also allows the Sista adaptive optimizer to inline FFI calls.
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
- A C interpreter plugin.
- A LLVM based backend.
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is missing
a better Slang generator, that allows to force better inlining and more
code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang.
Yes, and that's difficult because it's a moving target and I have been
lazy, not writing tests, instead using the Cog VM as "the test".

I am so happy to have your involvement. You and Cl?ment bring such
strength and competence.

---------------------------------------------------------------
Post by Clément Bera
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method that
is going to respond to the message. Then you have to inline the method. And
then you can start performing the actual optimizations, such as constant
folding, common subexpressions, dead branch elimination, loop unrolling,
and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
In other words, our best bet is in the work of Cl?ment in Sista. The only
problem with this bet are real time applications.
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.

Real time applications requires an upper bound guarantee in their response
Post by Clément Bera
time. In some cases, the lack of this guarantee can be just an annoyance,
as happens in video games. In some mission critical applications the
results can not be good, if this time constraint is not met. An example of
a mission critical system could the flight controls of an airplane, or the
cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive optimizer
- Extend the language to hand optimize some performance critical sections
of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But

a) I've yet to see anybody raise JIT latency as an issue in Cog
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.

And of course, you have to perform lot of profiling.
Early and often :-).

Because we can have complete control over the optimizer, and because Sista
is byetcode-to-bytecode and can hence store its results in the image in the
form of optimized methods, I believe that Sista is well-positioned for
real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.

Greetings,
Post by Clément Bera
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/aa6070b7/attachment-0001.html>
Thierry Goubier
2014-09-16 08:06:30 UTC
Permalink
Post by Eliot Miranda
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Great.

Would it also be possible from the outside to make sista-like
optimisations? I'm thinking of a few possibilities such as replacing some
dictionary constant building code and a few others (i.e. a kind of [ ] once
and the #(( )) of Dolphin) which could be handled at the user level (by
tracking if it has to be recompiled if someone, say, changes the
implementation of Dictionary).
Post by Eliot Miranda
Real time applications requires an upper bound guarantee in their response
Post by Ronie Salgado
time. In some cases, the lack of this guarantee can be just an annoyance,
as happens in video games. In some mission critical applications the
results can not be good, if this time constraint is not met. An example of
a mission critical system could the flight controls of an airplane, or the
cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical sections
of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
How many instructions you spend on average in Cog per generated instruction?
Post by Eliot Miranda
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
That one is interesting. It sounds like a great benefit would be to have a
limited API to the Cog JIT for that sort of requirements.
Post by Eliot Miranda
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and because Sista
is byetcode-to-bytecode and can hence store its results in the image in the
form of optimized methods, I believe that Sista is well-positioned for
real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
I'm not sure of that "real-time" aspect is a good idea. More real-time than
others dynamic optimisers, maybe. Note that Java Real-Time suggest turning
off JIT and switch to ITC.

The word you're looking for there is AoT Compiler :)

Regards,

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/a5c8c89b/attachment-0001.html>
Eliot Miranda
2014-09-16 22:12:35 UTC
Permalink
Hi Thierry,

On Tue, Sep 16, 2014 at 1:06 AM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by Eliot Miranda
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Great.
Would it also be possible from the outside to make sista-like
optimisations? I'm thinking of a few possibilities such as replacing some
dictionary constant building code and a few others (i.e. a kind of [ ] once
and the #(( )) of Dolphin) which could be handled at the user level (by
tracking if it has to be recompiled if someone, say, changes the
implementation of Dictionary).
There is no "outside" in Sista. It is an image-level optimizer, so you'll
be able to interact with it at the same level one interacts with the Opal
compiler. Of course doing this kind of strength-reduction is possible.


Real time applications requires an upper bound guarantee in their response
Post by Thierry Goubier
Post by Eliot Miranda
Post by Ronie Salgado
time. In some cases, the lack of this guarantee can be just an annoyance,
as happens in video games. In some mission critical applications the
results can not be good, if this time constraint is not met. An example of
a mission critical system could the flight controls of an airplane, or the
cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
How many instructions you spend on average in Cog per generated instruction?
I have no idea. All I know is that the code generation side of the Cogit
is extremely cheap, dwarfed by the costs of relinking when code is
recompiled. But the cost is vanishingly small. The costs that do show up
in the Cogit are the cost of discarding, compacting and relinking code when
the working set grows (e.g. Compiler recompileAll), and pause times here
are ~ 1ms on my 2.2GHz MBP; thats about twice the cost of a scavenge, at
about 1/20th the frequency (these times are for Spur). If you open up the
system reporter you'll be able to see for yourself. And if you use the
VMProfiler you can see exactly here the time goes for your favourite
synthetic benchmark. See if you can figure out how to create bytecoded
methods cheaper than the JIT can compile them and profile it.

If you've got a tool that can count instructions let me know and I may
measure. I could could bytecodes per instruction generated in the
simulator ;-).
Post by Thierry Goubier
b) it would be easy to extend the VM to cause the Cogit to precompile
Post by Eliot Miranda
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
That one is interesting. It sounds like a great benefit would be to have a
limited API to the Cog JIT for that sort of requirements.
And of course, you have to perform lot of profiling.
Post by Eliot Miranda
Early and often :-).
Because we can have complete control over the optimizer, and because
Sista is byetcode-to-bytecode and can hence store its results in the image
in the form of optimized methods, I believe that Sista is well-positioned
for real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
I'm not sure of that "real-time" aspect is a good idea. More real-time
than others dynamic optimisers, maybe. Note that Java Real-Time suggest
turning off JIT and switch to ITC.
But thats because in Java VMs the adaptive optimizer is in the VM where one
has limited control over it. Sista is different.
Post by Thierry Goubier
The word you're looking for there is AoT Compiler :)
In fact a better term would be an interactive compiler ;-)

Regards,
Post by Thierry Goubier
Thierry
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/a5897c4e/attachment-0001.html>
Thierry Goubier
2014-09-17 16:57:58 UTC
Permalink
Hi Eliot,
Post by Eliot Miranda
Hi Thierry,
On Tue, Sep 16, 2014 at 1:06 AM, Thierry Goubier
There is no "outside" in Sista. It is an image-level optimizer, so
you'll be able to interact with it at the same level one interacts with
the Opal compiler. Of course doing this kind of strength-reduction is
possible.
Ok. I'll have a look and maybe profile a bit; SmaCC has plenty of those
optimisations that I had to remove in the Pharo port, and I should have
a look on the cost of doing all thoses Character value: XXX and ^
Dictionary new...

I also had nice optimisations in VW for my trace tool (like replacing a
class lookup by a Literal which was then replaced by the singleton
instance in the method bytecode) which do not have any effect in
Squeak/Pharo anymore.
Post by Eliot Miranda
I have no idea. All I know is that the code generation side of the
Cogit is extremely cheap, dwarfed by the costs of relinking when code is
recompiled. But the cost is vanishingly small. The costs that do show
up in the Cogit are the cost of discarding, compacting and relinking
code when the working set grows (e.g. Compiler recompileAll), and pause
times here are ~ 1ms on my 2.2GHz MBP; thats about twice the cost of a
scavenge, at about 1/20th the frequency (these times are for Spur). If
you open up the system reporter you'll be able to see for yourself. And
if you use the VMProfiler you can see exactly here the time goes for
your favourite synthetic benchmark. See if you can figure out how to
create bytecoded methods cheaper than the JIT can compile them and
profile it.
I have colleagues in the same department working on localized, template
based JiT of computational kernels and they do 3 inst of JiT for one
instruction (and prove that they go faster on some matrix math code,
including the JiT overhead, than anything statically optimised off-line).

At the same time, a one ms pause at 2.2GHz is a hell of a long time if
your target is a MicroBlaze at 50 Mhz or an embedded powerpc at 25 MHz.
Post by Eliot Miranda
If you've got a tool that can count instructions let me know and I may
measure. I could could bytecodes per instruction generated in the
simulator ;-).
I used some of the intel cycle CPU counters in the past to benchmark
code. I'll ask around for accessing Intel's cycle count registers. I'm
sure the Bochs x86 simulator has those.
Post by Eliot Miranda
b) it would be easy to extend the VM to cause the Cogit to
precompile specified methods. We could easily provide a
"lock-down" facility that would prevent Cog from discarding
specific machine code methods.
That one is interesting. It sounds like a great benefit would be to
have a limited API to the Cog JIT for that sort of requirements.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and
because Sista is byetcode-to-bytecode and can hence store its
results in the image in the form of optimized methods, I believe
that Sista is well-positioned for real-time since it can be used
before deployment. In fact we should emphasise this in the
papers we write on Sista.
I'm not sure of that "real-time" aspect is a good idea. More
real-time than others dynamic optimisers, maybe. Note that Java
Real-Time suggest turning off JIT and switch to ITC.
But thats because in Java VMs the adaptive optimizer is in the VM where
one has limited control over it. Sista is different.
The problem is that even a simple JiT is considered too much Jitter :)

The Ocaml guys told once they had to do a special, interpreted VM for a
real-time use case. The problem is not going fast, the problem is being
predictable. They turn off CPU caches in some cases.

Thierry
Eliot Miranda
2014-09-17 17:18:37 UTC
Permalink
Hi Thierry,

On Wed, Sep 17, 2014 at 9:57 AM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Hi Eliot,
Post by Eliot Miranda
Hi Thierry,
On Tue, Sep 16, 2014 at 1:06 AM, Thierry Goubier
There is no "outside" in Sista. It is an image-level optimizer, so
you'll be able to interact with it at the same level one interacts with
the Opal compiler. Of course doing this kind of strength-reduction is
possible.
Ok. I'll have a look and maybe profile a bit; SmaCC has plenty of those
optimisations that I had to remove in the Pharo port, and I should have a
look on the cost of doing all thoses Character value: XXX and ^ Dictionary
new...
I also had nice optimisations in VW for my trace tool (like replacing a
class lookup by a Literal which was then replaced by the singleton instance
in the method bytecode) which do not have any effect in Squeak/Pharo
anymore.
Yes, and various dialects have a literal constructor for "compile-time
expressions" (I did one for VW) that provide a manual (and hence fragile)
way of doing this. As you pointed out this approach breaks when one
changes the code previously compiled. Having Sista regenerate for you
automatically is of course preferrable. But what do you mean "do not have
any effect in Squeak/Pharo anymore"?


I have no idea. All I know is that the code generation side of the
Post by Thierry Goubier
Post by Eliot Miranda
Cogit is extremely cheap, dwarfed by the costs of relinking when code is
recompiled. But the cost is vanishingly small. The costs that do show
up in the Cogit are the cost of discarding, compacting and relinking
code when the working set grows (e.g. Compiler recompileAll), and pause
times here are ~ 1ms on my 2.2GHz MBP; thats about twice the cost of a
scavenge, at about 1/20th the frequency (these times are for Spur). If
you open up the system reporter you'll be able to see for yourself. And
if you use the VMProfiler you can see exactly here the time goes for
your favourite synthetic benchmark. See if you can figure out how to
create bytecoded methods cheaper than the JIT can compile them and
profile it.
I have colleagues in the same department working on localized, template
based JiT of computational kernels and they do 3 inst of JiT for one
instruction (and prove that they go faster on some matrix math code,
including the JiT overhead, than anything statically optimised off-line).
Sure. My JIT optimizes and cannot be reduced to a template approach. But
it is still extremely fast. As I said, scanning code to relink/compact/GC
is what takes the time, not compilation itself. And the pause times here
are bounded (see below).

At the same time, a one ms pause at 2.2GHz is a hell of a long time if your
Post by Thierry Goubier
target is a MicroBlaze at 50 Mhz or an embedded powerpc at 25 MHz.
Sure, there are bound to be contexts in which Cog can't meet the required
deadlines. But that doesn't imply it can never do real time, does it? And
processor speeds are climbing. Look at Raspberry Pi, 700MHz. Not too
shabby.


If you've got a tool that can count instructions let me know and I may
Post by Thierry Goubier
Post by Eliot Miranda
measure. I could could bytecodes per instruction generated in the
simulator ;-).
I used some of the intel cycle CPU counters in the past to benchmark code.
I'll ask around for accessing Intel's cycle count registers. I'm sure the
Bochs x86 simulator has those.
b) it would be easy to extend the VM to cause the Cogit to
Post by Eliot Miranda
precompile specified methods. We could easily provide a
"lock-down" facility that would prevent Cog from discarding
specific machine code methods.
That one is interesting. It sounds like a great benefit would be to
have a limited API to the Cog JIT for that sort of requirements.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and
because Sista is byetcode-to-bytecode and can hence store its
results in the image in the form of optimized methods, I believe
that Sista is well-positioned for real-time since it can be used
before deployment. In fact we should emphasise this in the
papers we write on Sista.
I'm not sure of that "real-time" aspect is a good idea. More
real-time than others dynamic optimisers, maybe. Note that Java
Real-Time suggest turning off JIT and switch to ITC.
But thats because in Java VMs the adaptive optimizer is in the VM where
one has limited control over it. Sista is different.
The problem is that even a simple JiT is considered too much Jitter :)
But some in some contexts. But that doesn't define the entire real time
world.

The Ocaml guys told once they had to do a special, interpreted VM for a
Post by Thierry Goubier
real-time use case. The problem is not going fast, the problem is being
predictable. They turn off CPU caches in some cases.
Sure. But if one can show that the JIT's pause times are bounded and well
spaced then there is no fundamental problem. People have been implementing
real-time GCs and using GC in real time systems for over 20 years now.
This is no different. But you seem absolute in denying that there is any
possibility of using a JIT in real time. I think that claim is clearly
false.
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140917/e2cc105c/attachment.html>
Thierry Goubier
2014-09-17 18:46:38 UTC
Permalink
Hi Eliot
Post by Eliot Miranda
Hi Thierry,
Yes, and various dialects have a literal constructor for "compile-time
expressions" (I did one for VW) that provide a manual (and hence
fragile) way of doing this. As you pointed out this approach breaks
when one changes the code previously compiled. Having Sista regenerate
for you automatically is of course preferrable. But what do you mean
"do not have any effect in Squeak/Pharo anymore"?
It works but has no effect on performance. Well, this is an
optimisation; you gain in certain circonstances, you loose in others and
you turn it off.
Post by Eliot Miranda
I have colleagues in the same department working on localized,
template based JiT of computational kernels and they do 3 inst of
JiT for one instruction (and prove that they go faster on some
matrix math code, including the JiT overhead, than anything
statically optimised off-line).
Sure. My JIT optimizes and cannot be reduced to a template approach.
But it is still extremely fast. As I said, scanning code to
relink/compact/GC is what takes the time, not compilation itself. And
the pause times here are bounded (see below).
At the same time, a one ms pause at 2.2GHz is a hell of a long time
if your target is a MicroBlaze at 50 Mhz or an embedded powerpc at
25 MHz.
Sure, there are bound to be contexts in which Cog can't meet the
required deadlines. But that doesn't imply it can never do real time,
does it? And processor speeds are climbing. Look at Raspberry Pi,
700MHz. Not too shabby.
This is the performance of an old desktop :) And a not so old at that.
ARM started as a desktop processor, after all.

But, look: Python works very well on the Pi ;) Fast enough, and if not,
you call a C lib.
Post by Eliot Miranda
The problem is that even a simple JiT is considered too much Jitter :)
But some in some contexts. But that doesn't define the entire real time
world.
Yes.
Post by Eliot Miranda
The Ocaml guys told once they had to do a special, interpreted VM
for a real-time use case. The problem is not going fast, the problem
is being predictable. They turn off CPU caches in some cases.
Sure. But if one can show that the JIT's pause times are bounded and
well spaced then there is no fundamental problem. People have been
implementing real-time GCs and using GC in real time systems for over 20
years now. This is no different. But you seem absolute in denying that
there is any possibility of using a JIT in real time. I think that
claim is clearly false.
If we consider that you would have to do a WCET analysis / stack
boundedness analysis to prove that the JIT pauses are deterministic,
bounded and are schedulable for the target times (i.e. suitable for a
significant part of what real time is about), then that claim will hold
true for a while.

If we change and adjust the definition of real-time and the required
processing power needed, then we can consider Cog JIT sufficiently
real-time as long as it is reasonably deterministic.

This is my opinion on that: for what I have done in the embedded world
and on FPGA so far, any gain on the determinism, performance and
portability of Cog is highly significant(*); but I would really like
that the language running on top of Cog, and Cog, be able to clearly let
a programmer specify bounds / determinism requirements / parallelism so
as to ensure that it may communicate well with real-time programs. And
then it would be a really nice fit, and a huge boost to both Cog and the
Embedded/RT community.

(*) And those benefits would also make it really cool to use everywhere.

(**) A nice target would be to be able to run Cog on a SoC with a
general purpose CPU core and a FPGA.

(***) and then I could reuse what I know about synthesizing logic
circuits on a FPGA from smalltalk code :)

Regards,

Thierry
Eliot Miranda
2014-09-18 20:25:10 UTC
Permalink
Hi Thierry,

On Wed, Sep 17, 2014 at 11:46 AM, Thierry Goubier <thierry.goubier at gmail.com
Post by Thierry Goubier
Hi Eliot
Post by Eliot Miranda
Hi Thierry,
Yes, and various dialects have a literal constructor for "compile-time
expressions" (I did one for VW) that provide a manual (and hence
fragile) way of doing this. As you pointed out this approach breaks
when one changes the code previously compiled. Having Sista regenerate
for you automatically is of course preferrable. But what do you mean
"do not have any effect in Squeak/Pharo anymore"?
It works but has no effect on performance. Well, this is an optimisation;
you gain in certain circonstances, you loose in others and you turn it off.
Post by Eliot Miranda
I have colleagues in the same department working on localized,
template based JiT of computational kernels and they do 3 inst of
JiT for one instruction (and prove that they go faster on some
matrix math code, including the JiT overhead, than anything
statically optimised off-line).
Sure. My JIT optimizes and cannot be reduced to a template approach.
But it is still extremely fast. As I said, scanning code to
relink/compact/GC is what takes the time, not compilation itself. And
the pause times here are bounded (see below).
At the same time, a one ms pause at 2.2GHz is a hell of a long time
if your target is a MicroBlaze at 50 Mhz or an embedded powerpc at
25 MHz.
Sure, there are bound to be contexts in which Cog can't meet the
required deadlines. But that doesn't imply it can never do real time,
does it? And processor speeds are climbing. Look at Raspberry Pi,
700MHz. Not too shabby.
This is the performance of an old desktop :) And a not so old at that. ARM
started as a desktop processor, after all.
But, look: Python works very well on the Pi ;) Fast enough, and if not,
you call a C lib.
and that's precisely what we want to avoid. IMO that;s one reason why the
Python (and e.g. PHP) community has been so slow to implement good VMs. It
has deferred to C code for performance-critical code _where it could_ (e.g.
regular expressions) at the expense of rigid dependence on that external
library. Now that companies such as Dropbox and Facebook are needing
Python and PHP to exhibit good performance they are finally implementing
faster VMs long after other languages. This has meant that Python
libraries have not evolved as fast or as conveniently as could have
happened if there was no dependency on C.

Of course the situation with Squeak was/is the same, with a proliferation
of plugins that would not be necessary if the VM were fast. (of course some
plugins exist for good security reasons).


The problem is that even a simple JiT is considered too much Jitter :)
Post by Thierry Goubier
Post by Eliot Miranda
But some in some contexts. But that doesn't define the entire real time
world.
Yes.
Post by Eliot Miranda
The Ocaml guys told once they had to do a special, interpreted VM
for a real-time use case. The problem is not going fast, the problem
is being predictable. They turn off CPU caches in some cases.
Sure. But if one can show that the JIT's pause times are bounded and
well spaced then there is no fundamental problem. People have been
implementing real-time GCs and using GC in real time systems for over 20
years now. This is no different. But you seem absolute in denying that
there is any possibility of using a JIT in real time. I think that
claim is clearly false.
If we consider that you would have to do a WCET analysis / stack
boundedness analysis to prove that the JIT pauses are deterministic,
bounded and are schedulable for the target times (i.e. suitable for a
significant part of what real time is about), then that claim will hold
true for a while.
Right.

If we change and adjust the definition of real-time and the required
Post by Thierry Goubier
processing power needed, then we can consider Cog JIT sufficiently
real-time as long as it is reasonably deterministic.
This is my opinion on that: for what I have done in the embedded world and
on FPGA so far, any gain on the determinism, performance and portability of
Cog is highly significant(*); but I would really like that the language
running on top of Cog, and Cog, be able to clearly let a programmer specify
bounds / determinism requirements / parallelism so as to ensure that it may
communicate well with real-time programs. And then it would be a really
nice fit, and a huge boost to both Cog and the Embedded/RT community.
(*) And those benefits would also make it really cool to use everywhere.
(**) A nice target would be to be able to run Cog on a SoC with a general
purpose CPU core and a FPGA.
(***) and then I could reuse what I know about synthesizing logic circuits
on a FPGA from smalltalk code :)
I look forward to working with you on this! We may be a few years behind
this curve in first getting Sista and the FFI and 64-bits done, but soon
enough we will be ready to tackle challenges like this!

Regards,
Post by Thierry Goubier
Thierry
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140918/9be63b10/attachment.html>
kilon alios
2014-09-18 21:25:30 UTC
Permalink
" and that's precisely what we want to avoid. IMO that;s one reason why
the Python (and e.g. PHP) community has been so slow to implement good VMs.
It has deferred to C code for performance-critical code _where it could_
(e.g. regular expressions) at the expense of rigid dependence on that
external library. Now that companies such as Dropbox and Facebook are
needing Python and PHP to exhibit good performance they are finally
implementing faster VMs long after other languages. "

nope , you are close but you got it the other way around. As a smalltalker
you see it from purist way of view. For you Python is that language that
got caught up on too reliance on C/C++ libraries.

What you see as side effect is actually the central goal of python. The
central goal of python was and still is to a great extend today to serve as
scripting language for C and C++ code. This what made python so successful
that a C/C++ coder could use on top of its app embedded inside its app.
This why so many people use it even today.

You also again wrong to assume that its a late thing that new VMs are
developed for Python , PyPy which is very well known inside the python
community has been around more than 12 years. Its a very performant JIT VM
2-10 faster than cpython. Why cpython has not ported to it ? because of not
so good support for C/C++ , zero support for cpython libraries and of
course because as they say it would complicate the development of cpython
quite a lot . Its that situation that only a couple of people can decypher
the magic of the VM and the rest of developer only hope for the best.

And PyPy is not the only JIT VM for python , there is also Jython that runs
of JVM and ironpython that runs on .NET and Mono. What those 3 have in
common ?that are used by a tiny crowd compared to people using cpython.

The truth is pure python libraries are a rarity . Why you think the most
popular , by far, implementation of python is called cpython ? I will give
you a hint, 50% of its source code is written in C. No thats not just the
VM its libraries and cpython comes with a lot of libraries included.

Frankly I dont believe that any VM ever however fast will ever beat smooth
integration of C/C++ code unless you have a huge company to back you up
with an insane amount of libraries. Java had Sun , .NET has still
Microsoft.

For people that speed is a serious matter even if a VM is a mere 30% slower
than optimised C code, they will take the C code any day.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140919/fadceee7/attachment.html>
Eliot Miranda
2014-09-18 22:39:37 UTC
Permalink
Post by kilon alios
" and that's precisely what we want to avoid. IMO that;s one reason why
the Python (and e.g. PHP) community has been so slow to implement good VMs.
It has deferred to C code for performance-critical code _where it could_
(e.g. regular expressions) at the expense of rigid dependence on that
external library. Now that companies such as Dropbox and Facebook are
needing Python and PHP to exhibit good performance they are finally
implementing faster VMs long after other languages. "
nope , you are close but you got it the other way around. As a smalltalker
you see it from purist way of view. For you Python is that language that
got caught up on too reliance on C/C++ libraries.
Nope. As a C/C++ advocate you are in denial about how unproductive your
tools are and about the fact that it takes industrial scale organization to
achieve even the most trivial of achievements.
Post by kilon alios
What you see as side effect is actually the central goal of python. The
central goal of python was and still is to a great extend today to serve as
scripting language for C and C++ code. This what made python so successful
that a C/C++ coder could use on top of its app embedded inside its app.
This why so many people use it even today.
Nope. What you claim as a central goal of Python is merely a supine
acceptance of massive failure to innovate. Bourne shell has been doing
this for years and years. It's not a new thing. And GVR seems to have
conceived it as a better shell. Well, that's not innovative; useful, yes,
innovative no.

But seriously, look at something innovative written in Python such as
BitTorrent et al. How much of that is system calls, C or Python? t is
basically Python an socket calls. C doesn't play a significant part in the
architecture. The important thing is a dynamic OOPL being able to express
a novel architecture. I'd wager that torrent clients are amongst the most
popular Python apps.

You also again wrong to assume that its a late thing that new VMs are
Post by kilon alios
developed for Python , PyPy which is very well known inside the python
community has been around more than 12 years. Its a very performant JIT VM
2-10 faster than cpython. Why cpython has not ported to it ? because of not
so good support for C/C++ , zero support for cpython libraries and of
course because as they say it would complicate the development of cpython
quite a lot . Its that situation that only a couple of people can decypher
the magic of the VM and the rest of developer only hope for the best.
By recently I mean within the last 10 years or so. The HotSpot Smalltalk
VM is already 19 years old, and Self 3 over 20. PS, the first truly fast
VM for a dynamic object oriented language is 32 years old. Python could
have started an implementation effort at a faster VM much sooner and it
would have shaped Python, IMO positively. There are reasons Java has been
much more popular until recently, performance being a very important one.
(There's reasons it's no longer popular, IMO the most important being it
being statically typed).

Further if you read what the Dropbox guys (and many others) are saying
about Pyston vs PyPy is that tracing JITs fail to deliver good performance
except in synthetic benchmarks where they shine, and hence Pyston is a
method-at-a-time JIT. So PyPy is a so-so VM, hardly in the same league as
HotSpot or V8 (and Pyston will have a ways to go before i can compete on
that level), just as Cog is a so-so VM and needs Sista to reach or exceed
static language performance for non-symbolic codes.

And the lack of good support for C/C++ in PyPy is not a technical
limitation. It is an organizational one. There's nothing intrinsic in a
fast VM that makes it more difficult to interface to C. An execution
engine is an execution engine, and it is the object representation that
determines how easy it is to interface to other languages.

And PyPy is not the only JIT VM for python , there is also Jython that runs
Post by kilon alios
of JVM and ironpython that runs on .NET and Mono. What those 3 have in
common ?that are used by a tiny crowd compared to people using cpython.
That could be because foreign VMs don't provide the support that users of a
VM designed for a language expect, could it not? IMO this is a really
important issue, especially for Smalltalk where we expect instance
migration, contexts, etc, all of which one has to say goodbye to when
hosted on a foreign VM. And this is a major reason for putting effort into
one's own infrastructure.

The truth is pure python libraries are a rarity . Why you think the most
Post by kilon alios
popular , by far, implementation of python is called cpython ? I will give
you a hint, 50% of its source code is written in C. No thats not just the
VM its libraries and cpython comes with a lot of libraries included.
And that pure python libraries are a rarity is of course a result of the
fact that Python has run to C whenever it needed performance. Hence when
one comes to measure the speedup of a fast python against cpython it is
difficult to see speedups because lots of benchmarks (e.g. many of the
computer language shootout benchmarks) are spending much of their time
running C code. Hint, if an app spends 50% of its time in C then it can
never be more than twice as fast on a faster execution engine.

Frankly I dont believe that any VM ever however fast will ever beat smooth
Post by kilon alios
integration of C/C++ code unless you have a huge company to back you up
with an insane amount of libraries. Java had Sun , .NET has still
Microsoft.
Seriously, what do you mean "beat"? The world is infinitely varied. COBOL
is still important and yet you seem to portray the entire programming field
as reducing to some kind of horse race. What's the content here? Also
what's the intent behind pissing contests? Seems rather teenage. Whereas
being positive, doing good engineering and innovating can invent the
future. Seem much more fun than doling out endless negativity.
Post by kilon alios
For people that speed is a serious matter even if a VM is a mere 30%
slower than optimised C code, they will take the C code any day.
You contradicted yourself in claiming that Pythoners dont yse faster VMs.
Java has good C integration and provides much better performance than PyPy
so if you were correct Pythoners would be using Jython + C but you claim
they're not. (Personally I'm dubious about blanket claims about what
people do and don't do; my experience since I've been involved with
commercial Smalltalk deployment is that the world is incredibly varied and
different people with different requirements do different things, often for
good reason).

Speed is pretty meaningless unless you qualify it. In applications where
performance can really matter theres always assembler (e.g. video decoders)
or DSPs or... and often optimized C just doesn't hack it. But so what?
There is another context where using Scheme to automatically generate Java
VMs is critical to performance. And one can cite example after example
where high performance does /not/ equal C/C++ plus an insane number of
libraries. Computing is a broad church.

There are plenty of contexts where time to market is the dominant factor
and that can force one to reject performant and difficult to configure
solutions (such as Jython, or Smalltalk + C), but equally there
are contexts where time to market is the dominant factor and that can
"force" one to use pure dynamic OOPLs because the world changes too fast to
do anything else (e.g. finance).

Anyway, keep those "nope"s coming. I love them. Time to patent your new
way of proving a negative I think.
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140918/ccf5bdf4/attachment-0001.html>
kilon alios
2014-09-18 23:45:57 UTC
Permalink
"Nope. As a C/C++ advocate you are in denial about how unproductive your
tools are and about the fact that it takes industrial scale organization to
achieve even the most trivial of achievements."

C/C++ , I don't code in those languages. Currently I use pharo, before that
I was coding in python and still do, before that I was a Delphi developer.
I hate C++ with a vengeance.I am now learning C for my first time.

I dont see the industry having any problem achieving what it wants with
C/C++ because if it did then the software industry would be overrun by
smalltalk , which we all know how superior better it is . Actually you will
be lucky if a coder was even aware what smalltalk is. I did not know what
smalltalk is like a few years ago. I never even saw it mentioned in coding
forums or in chat channels until one day someone mentioned squeak to me in
#emacs.

"By recently I mean within the last 10 years or so. The HotSpot Smalltalk
VM is already 19 years old, and Self 3 over 20. PS, the first truly fast
VM for a dynamic object oriented language is 32 years old. Python could
have started an implementation effort at a faster VM much sooner and it
would have shaped Python, IMO positively. There are reasons Java has been
much more popular until recently, performance being a very important one.
(There's reasons it's no longer popular, IMO the most important being it
being statically typed).

Further if you read what the Dropbox guys (and many others) are saying
about Pyston vs PyPy is that tracing JITs fail to deliver good performance
except in synthetic benchmarks where they shine, and hence Pyston is a
method-at-a-time JIT. So PyPy is a so-so VM, hardly in the same league as
HotSpot or V8 (and Pyston will have a ways to go before i can compete on
that level), just as Cog is a so-so VM and needs Sista to reach or exceed
static language performance for non-symbolic codes.

And the lack of good support for C/C++ in PyPy is not a technical
limitation. It is an organizational one. There's nothing intrinsic in a
fast VM that makes it more difficult to interface to C. An execution
engine is an execution engine, and it is the object representation that
determines how easy it is to interface to other languages."

Bottom line is clear, do optimised VMs are likely to really interest
people, from the looks of it does not seem so. I don't care whether PyPy
has technical limitations what matters is that I cannot use my favorite
cpython libraries with it.
"
And that pure python libraries are a rarity is of course a result of the
fact that Python has run to C whenever it needed performance. Hence when
one comes to measure the speedup of a fast python against cpython it is
difficult to see speedups because lots of benchmarks (e.g. many of the
computer language shootout benchmarks) are spending much of their time
running C code. Hint, if an app spends 50% of its time in C then it can
never be more than twice as fast on a faster execution engine."

no python needs to run C libraries because it wants to run C libraries ,
because they are so many out there and many of them are so damn good at
what they are doing. Actually nowadays it becomes increasingly difficult
not to find python wrappers for a C/C++ library. It comes down to C/C++
coders needing a simpler language that can easily port their code a lot
more than a python coder would need C for speed.

"You contradicted yourself in claiming that Pythoners dont yse faster VMs.
Java has good C integration and provides much better performance than PyPy
so if you were correct Pythoners would be using Jython + C but you claim
they're not. (Personally I'm dubious about blanket claims about what
people do and don't do; my experience since I've been involved with
commercial Smalltalk deployment is that the world is incredibly varied and
different people with different requirements do different things, often for
good reason)."

First of all Java has had crappy C integration like for ages , people were
complaining about it for a long long time. Second I dont see how I
contradict myself , I claimed that python people don't care so much about
how fast a VM is when they can fall back to C and you bring up Java speed.
What am I missing her ?

Why not use Jython ? simple same reason why people dont use PyPy lack of
support for cpython libraries, which by the way are based partly if not
largely in C and if Java had so cool support of C then jython would have
supported cpython libraries out of the box , which it does not.

We can run this in circles but in the end I find it highly suspicious how
some smalltalkers find smalltalk so awesome in every level yet they fail to
justify why its so unpopular.

And just for the record , so people dont think I talk out of random, this
is Guido , the creator of python , says about PyPy and why is not
integrated to cpython and the main goal of cpython -->


And yes python does not care about innovation at all, and this is one of
the reason of its immense success as a language. Actually if you take into
consideration that python is not tied to platform like javascript , or is
not backed up by a huge company like C++ , Java , C# I dont think its an
exaggeration at all that to say that it has been by far the most successful
language at least in terms of people actually using it.

Personally I find python as well completely unoriginal , not that well
implemented, very slow and unimaginative. Still its the language, my number
one choice, I would recommend to any one in terms of practicality alone.

I don't want to debate this to death, I stated my opinion and the my view
on things, you can call me advocate , fanboy or plain blind. Thats your
right. I think however its good to hear the other side of story , the other
opinion to make the discussion a bit more spherical.

Saying that I love smalltalk and especially pharo and I hope it keeps being
practical enough for me to keep using it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140919/e40f5387/attachment.html>
Clément Bera
2014-09-16 09:48:46 UTC
Permalink
Post by Eliot Miranda
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
remember to send me code for integration. I'm eagerly waiting to use your
code!
Lowcode is currently a spec of new bytecode instructions. These
Post by Clément Bera
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new bytecode
set for SistaV1, which is called "inline primitives". Because of this,
these new instructions can be mixed freely with the standard VM bytecode
set. This also allows the Sista adaptive optimizer to inline FFI calls.
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
- A C interpreter plugin.
- A LLVM based backend.
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is missing
a better Slang generator, that allows to force better inlining and more
code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang.
Lack of documentation ?

About Cog there are these documentation:
Back to the future <http://ftp.squeak.org/docs/OOPSLA.Squeak.html>
About VMMaker <http://wiki.squeak.org/squeak/2105>
Object engine
<http://www.rowledge.org/resources/tim%27s-Home-page/Squeak/OE-Tour.pdf>
General information <http://squeakvm.org/index.html>
Blue book part 4
<http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf>
Deep into Pharo part 4 about blocks and exceptions
<http://www.deepintopharo.com/>
VMIL paper about Cogit
<http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf>
The Cog blog <http://www.mirandabanda.org/cogblog/>
About Spur: summary
<http://clementbera.wordpress.com/2014/02/06/7-points-summary-of-the-spur-memory-manager/>
and
object format
<http://clementbera.wordpress.com/2014/01/16/spurs-new-object-format/>
This post <http://clementbera.wordpress.com/2013/08/09/the-cog-vm-lookup/>
And many useful class and method comments that taught me a lot.

When I try to work with Pharo frameworks, even recent ones, it is very rare
that I see as much documentation than it exists for Cog. Some frameworks
are documented in the Pharo books and a few other as Zinc have good
documentation, but in general, there are few documentation and *even fewer
people writing documentation*. The website about Cog has existed for over 6
years now. I think Cog is far from the worst documented part of Pharo.
Post by Eliot Miranda
Yes, and that's difficult because it's a moving target and I have been
lazy, not writing tests, instead using the Cog VM as "the test".
It's also difficult because the first tests to write are the hardest to
write.

I am so happy to have your involvement. You and Cl?ment bring such
Post by Eliot Miranda
strength and competence.
---------------------------------------------------------------
Post by Clément Bera
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations, such as
constant folding, common subexpressions, dead branch elimination, loop
unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
In other words, our best bet is in the work of Cl?ment in Sista. The only
problem with this bet are real time applications.
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Real time applications requires an upper bound guarantee in their response
Post by Clément Bera
time. In some cases, the lack of this guarantee can be just an annoyance,
as happens in video games. In some mission critical applications the
results can not be good, if this time constraint is not met. An example of
a mission critical system could the flight controls of an airplane, or the
cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical sections
of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and because Sista
is byetcode-to-bytecode and can hence store its results in the image in the
form of optimized methods, I believe that Sista is well-positioned for
real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
The solution of Eliot makes sense.
To write a paper about that I need benchs showing result on real time
applications.
So there's quite some work to do before.
Post by Eliot Miranda
Greetings,
Post by Clément Bera
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/e617f496/attachment-0001.html>
kilon alios
2014-09-16 12:07:54 UTC
Permalink
Is there any other documentation for Slang apart from the link in squeak
wiki ? Hows Slang deals with manual memory management ?

On Tue, Sep 16, 2014 at 12:48 PM, Cl?ment Bera <bera.clement at gmail.com>
Post by Clément Bera
Post by Eliot Miranda
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
remember to send me code for integration. I'm eagerly waiting to use
your code!
Lowcode is currently a spec of new bytecode instructions. These
Post by Clément Bera
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new bytecode
set for SistaV1, which is called "inline primitives". Because of this,
these new instructions can be mixed freely with the standard VM bytecode
set. This also allows the Sista adaptive optimizer to inline FFI calls.
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
- A C interpreter plugin.
- A LLVM based backend.
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is
missing a better Slang generator, that allows to force better inlining and
more code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang.
Lack of documentation ?
Back to the future <http://ftp.squeak.org/docs/OOPSLA.Squeak.html>
About VMMaker <http://wiki.squeak.org/squeak/2105>
Object engine
<http://www.rowledge.org/resources/tim%27s-Home-page/Squeak/OE-Tour.pdf>
General information <http://squeakvm.org/index.html>
Blue book part 4
<http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf>
Deep into Pharo part 4 about blocks and exceptions
<http://www.deepintopharo.com/>
VMIL paper about Cogit
<http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf>
The Cog blog <http://www.mirandabanda.org/cogblog/>
About Spur: summary
<http://clementbera.wordpress.com/2014/02/06/7-points-summary-of-the-spur-memory-manager/> and
object format
<http://clementbera.wordpress.com/2014/01/16/spurs-new-object-format/>
This post <http://clementbera.wordpress.com/2013/08/09/the-cog-vm-lookup/>
And many useful class and method comments that taught me a lot.
When I try to work with Pharo frameworks, even recent ones, it is very
rare that I see as much documentation than it exists for Cog. Some
frameworks are documented in the Pharo books and a few other as Zinc have
good documentation, but in general, there are few documentation and *even
fewer people writing documentation*. The website about Cog has existed
for over 6 years now. I think Cog is far from the worst documented part of
Pharo.
Post by Eliot Miranda
Yes, and that's difficult because it's a moving target and I have been
lazy, not writing tests, instead using the Cog VM as "the test".
It's also difficult because the first tests to write are the hardest to
write.
I am so happy to have your involvement. You and Cl?ment bring such
Post by Eliot Miranda
strength and competence.
---------------------------------------------------------------
Post by Clément Bera
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations, such as
constant folding, common subexpressions, dead branch elimination, loop
unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
In other words, our best bet is in the work of Cl?ment in Sista. The
only problem with this bet are real time applications.
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Real time applications requires an upper bound guarantee in their
Post by Clément Bera
response time. In some cases, the lack of this guarantee can be just an
annoyance, as happens in video games. In some mission critical applications
the results can not be good, if this time constraint is not met. An example
of a mission critical system could the flight controls of an airplane, or
the cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and because
Sista is byetcode-to-bytecode and can hence store its results in the image
in the form of optimized methods, I believe that Sista is well-positioned
for real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
The solution of Eliot makes sense.
To write a paper about that I need benchs showing result on real time
applications.
So there's quite some work to do before.
Post by Eliot Miranda
Greetings,
Post by Clément Bera
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/a031b6b9/attachment-0001.html>
Eliot Miranda
2014-09-16 22:24:44 UTC
Permalink
Hi Kilon,
Post by kilon alios
Is there any other documentation for Slang apart from the link in squeak
wiki ?
Read the VM sources; they're the best example of what it does. But alas
documentation is poor here. You might read Lazy Become and Primitives
<http://www.mirandabanda.org/cogblog/2014/02/08/primitives-and-the-partial-read-barrier/>
as
an example of what one can do.
Post by kilon alios
Hows Slang deals with manual memory management ?
Poorly. Essentially one uses cCode: [.....] inSmalltalk: [...] to separate
the solutions used in each realm. For example, in the Cog JIT arrays of
instructions, branch fixups and stack entry descriptors are the key data
structures in code generation. In the Simulator these are Smalltalk
objects, and the Smalltalk GC reclaims memory. In the C VM these are stack
allocated via alloca and reclaimed on return. Further, Spur supports
pinned objects on the heap which the system can use to allocate memory in
place of malloc (the advantage being that the Simulator can use this memory
also). The simulator *could* use malloc if it contained a malloc
simulation, but as yet I haven't needed it, and given Spur's support for
pinned objects it is not compelling to implement a malloc simulation.
Post by kilon alios
On Tue, Sep 16, 2014 at 12:48 PM, Cl?ment Bera <bera.clement at gmail.com>
Post by Clément Bera
Post by Eliot Miranda
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
remember to send me code for integration. I'm eagerly waiting to use
your code!
Lowcode is currently a spec of new bytecode instructions. These
Post by Clément Bera
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new
bytecode set for SistaV1, which is called "inline primitives". Because of
this, these new instructions can be mixed freely with the standard VM
bytecode set. This also allows the Sista adaptive optimizer to inline FFI
calls.
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
- A C interpreter plugin.
- A LLVM based backend.
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is
missing a better Slang generator, that allows to force better inlining and
more code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang.
Lack of documentation ?
Back to the future <http://ftp.squeak.org/docs/OOPSLA.Squeak.html>
About VMMaker <http://wiki.squeak.org/squeak/2105>
Object engine
<http://www.rowledge.org/resources/tim%27s-Home-page/Squeak/OE-Tour.pdf>
General information <http://squeakvm.org/index.html>
Blue book part 4
<http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf>
Deep into Pharo part 4 about blocks and exceptions
<http://www.deepintopharo.com/>
VMIL paper about Cogit
<http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf>
The Cog blog <http://www.mirandabanda.org/cogblog/>
About Spur: summary
<http://clementbera.wordpress.com/2014/02/06/7-points-summary-of-the-spur-memory-manager/> and
object format
<http://clementbera.wordpress.com/2014/01/16/spurs-new-object-format/>
This post
<http://clementbera.wordpress.com/2013/08/09/the-cog-vm-lookup/>
And many useful class and method comments that taught me a lot.
When I try to work with Pharo frameworks, even recent ones, it is very
rare that I see as much documentation than it exists for Cog. Some
frameworks are documented in the Pharo books and a few other as Zinc have
good documentation, but in general, there are few documentation and *even
fewer people writing documentation*. The website about Cog has existed
for over 6 years now. I think Cog is far from the worst documented part of
Pharo.
Post by Eliot Miranda
Yes, and that's difficult because it's a moving target and I have been
lazy, not writing tests, instead using the Cog VM as "the test".
It's also difficult because the first tests to write are the hardest to
write.
I am so happy to have your involvement. You and Cl?ment bring such
Post by Eliot Miranda
strength and competence.
---------------------------------------------------------------
Post by Clément Bera
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations, such as
constant folding, common subexpressions, dead branch elimination, loop
unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
In other words, our best bet is in the work of Cl?ment in Sista. The
only problem with this bet are real time applications.
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Real time applications requires an upper bound guarantee in their
Post by Clément Bera
response time. In some cases, the lack of this guarantee can be just an
annoyance, as happens in video games. In some mission critical applications
the results can not be good, if this time constraint is not met. An example
of a mission critical system could the flight controls of an airplane, or
the cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and because
Sista is byetcode-to-bytecode and can hence store its results in the image
in the form of optimized methods, I believe that Sista is well-positioned
for real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
The solution of Eliot makes sense.
To write a paper about that I need benchs showing result on real time
applications.
So there's quite some work to do before.
Post by Eliot Miranda
Greetings,
Post by Clément Bera
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
--
best,
Eliot
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/d1770737/attachment-0001.html>
phil
2014-09-16 12:55:39 UTC
Permalink
What would be valuable is a reading list / path to VM enlightenment.

Bluebook is useful
Then a tour of the Object Engine by Tim
Then plugin articles + Slang
The bytecode set
Primitive...
Context to stack mapping
Blocks
Non local returns
Display/Sensor/event look/timer implementation (like in the porting
document).
and only then one would move to more advanced topics.

I saw that Clement had a set of VM related books on his desk at INRIA,
maybe posting the list would be great!

All the best,
Phil



On Tue, Sep 16, 2014 at 11:48 AM, Cl?ment Bera <bera.clement at gmail.com>
Post by Clément Bera
Post by Eliot Miranda
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
remember to send me code for integration. I'm eagerly waiting to use
your code!
Lowcode is currently a spec of new bytecode instructions. These
Post by Clément Bera
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new bytecode
set for SistaV1, which is called "inline primitives". Because of this,
these new instructions can be mixed freely with the standard VM bytecode
set. This also allows the Sista adaptive optimizer to inline FFI calls.
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
- A C interpreter plugin.
- A LLVM based backend.
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is
missing a better Slang generator, that allows to force better inlining and
more code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang.
Lack of documentation ?
Back to the future <http://ftp.squeak.org/docs/OOPSLA.Squeak.html>
About VMMaker <http://wiki.squeak.org/squeak/2105>
Object engine
<http://www.rowledge.org/resources/tim%27s-Home-page/Squeak/OE-Tour.pdf>
General information <http://squeakvm.org/index.html>
Blue book part 4
<http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf>
Deep into Pharo part 4 about blocks and exceptions
<http://www.deepintopharo.com/>
VMIL paper about Cogit
<http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf>
The Cog blog <http://www.mirandabanda.org/cogblog/>
About Spur: summary
<http://clementbera.wordpress.com/2014/02/06/7-points-summary-of-the-spur-memory-manager/> and
object format
<http://clementbera.wordpress.com/2014/01/16/spurs-new-object-format/>
This post <http://clementbera.wordpress.com/2013/08/09/the-cog-vm-lookup/>
And many useful class and method comments that taught me a lot.
When I try to work with Pharo frameworks, even recent ones, it is very
rare that I see as much documentation than it exists for Cog. Some
frameworks are documented in the Pharo books and a few other as Zinc have
good documentation, but in general, there are few documentation and *even
fewer people writing documentation*. The website about Cog has existed
for over 6 years now. I think Cog is far from the worst documented part of
Pharo.
Post by Eliot Miranda
Yes, and that's difficult because it's a moving target and I have been
lazy, not writing tests, instead using the Cog VM as "the test".
It's also difficult because the first tests to write are the hardest to
write.
I am so happy to have your involvement. You and Cl?ment bring such
Post by Eliot Miranda
strength and competence.
---------------------------------------------------------------
Post by Clément Bera
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations, such as
constant folding, common subexpressions, dead branch elimination, loop
unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
In other words, our best bet is in the work of Cl?ment in Sista. The
only problem with this bet are real time applications.
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Real time applications requires an upper bound guarantee in their
Post by Clément Bera
response time. In some cases, the lack of this guarantee can be just an
annoyance, as happens in video games. In some mission critical applications
the results can not be good, if this time constraint is not met. An example
of a mission critical system could the flight controls of an airplane, or
the cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and because
Sista is byetcode-to-bytecode and can hence store its results in the image
in the form of optimized methods, I believe that Sista is well-positioned
for real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
The solution of Eliot makes sense.
To write a paper about that I need benchs showing result on real time
applications.
So there's quite some work to do before.
Post by Eliot Miranda
Greetings,
Post by Clément Bera
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/fd1b9c99/attachment-0001.html>
Clément Bera
2014-09-16 13:34:16 UTC
Permalink
Post by phil
What would be valuable is a reading list / path to VM enlightenment.
Bluebook is useful
Then a tour of the Object Engine by Tim
Then plugin articles + Slang
The bytecode set
Primitive...
Context to stack mapping
Blocks
Non local returns
Display/Sensor/event look/timer implementation (like in the porting
document).
and only then one would move to more advanced topics.
I saw that Clement had a set of VM related books on his desk at INRIA,
maybe posting the list would be great!
The book that explains the best how to implement a high performance VM for
Smalltalk and why is Urs Holzle phd
<http://www.cs.ucsb.edu/~urs/oocsb/self/papers/urs-thesis.html>.

Other relevant books on my office focus on specific topics, such
as Advanced Compiler Design and Implementation by Steven Muchnick for
optimizing compilers or The garbage collection handbook by Richard Jones,
Antony Hosking and Eliot Moss.

All the best,
Post by phil
Phil
On Tue, Sep 16, 2014 at 11:48 AM, Cl?ment Bera <bera.clement at gmail.com>
Post by Clément Bera
Post by Eliot Miranda
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
remember to send me code for integration. I'm eagerly waiting to use
your code!
Lowcode is currently a spec of new bytecode instructions. These
Post by Clément Bera
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new
bytecode set for SistaV1, which is called "inline primitives". Because of
this, these new instructions can be mixed freely with the standard VM
bytecode set. This also allows the Sista adaptive optimizer to inline FFI
calls.
- Int32 and Int64 integer arithmetic without type checking.
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
- A C interpreter plugin.
- A LLVM based backend.
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is
missing a better Slang generator, that allows to force better inlining and
more code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang.
Lack of documentation ?
Back to the future <http://ftp.squeak.org/docs/OOPSLA.Squeak.html>
About VMMaker <http://wiki.squeak.org/squeak/2105>
Object engine
<http://www.rowledge.org/resources/tim%27s-Home-page/Squeak/OE-Tour.pdf>
General information <http://squeakvm.org/index.html>
Blue book part 4
<http://stephane.ducasse.free.fr/FreeBooks/BlueBook/Bluebook.pdf>
Deep into Pharo part 4 about blocks and exceptions
<http://www.deepintopharo.com/>
VMIL paper about Cogit
<http://design.cs.iastate.edu/vmil/2011/papers/p03-miranda.pdf>
The Cog blog <http://www.mirandabanda.org/cogblog/>
About Spur: summary
<http://clementbera.wordpress.com/2014/02/06/7-points-summary-of-the-spur-memory-manager/> and
object format
<http://clementbera.wordpress.com/2014/01/16/spurs-new-object-format/>
This post
<http://clementbera.wordpress.com/2013/08/09/the-cog-vm-lookup/>
And many useful class and method comments that taught me a lot.
When I try to work with Pharo frameworks, even recent ones, it is very
rare that I see as much documentation than it exists for Cog. Some
frameworks are documented in the Pharo books and a few other as Zinc have
good documentation, but in general, there are few documentation and *even
fewer people writing documentation*. The website about Cog has existed
for over 6 years now. I think Cog is far from the worst documented part of
Pharo.
Post by Eliot Miranda
Yes, and that's difficult because it's a moving target and I have been
lazy, not writing tests, instead using the Cog VM as "the test".
It's also difficult because the first tests to write are the hardest to
write.
I am so happy to have your involvement. You and Cl?ment bring such
Post by Eliot Miranda
strength and competence.
---------------------------------------------------------------
Post by Clément Bera
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations, such as
constant folding, common subexpressions, dead branch elimination, loop
unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
In other words, our best bet is in the work of Cl?ment in Sista. The
only problem with this bet are real time applications.
Ah! But! Sista has an advantage that other adaptive optimizers don't.
Because it optimizes from bytecode to bytecode it can be used during a
training phase and then switched off.
Real time applications requires an upper bound guarantee in their
Post by Clément Bera
response time. In some cases, the lack of this guarantee can be just an
annoyance, as happens in video games. In some mission critical applications
the results can not be good, if this time constraint is not met. An example
of a mission critical system could the flight controls of an airplane, or
the cooling system of a nuclear reactor.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
The additional option is to "train" the optimizer by running the
application before deploying and capturing the optimised methods. Discuss
this with Cl?ment and he'll explain how straight-forward it should be.
This still leaves the latency in the Cogit when it compiles from bytecode
to machine code. But
a) I've yet to see anybody raise JIT latency as an issue in Cog
b) it would be easy to extend the VM to cause the Cogit to precompile
specified methods. We could easily provide a "lock-down" facility that
would prevent Cog from discarding specific machine code methods.
And of course, you have to perform lot of profiling.
Early and often :-).
Because we can have complete control over the optimizer, and because
Sista is byetcode-to-bytecode and can hence store its results in the image
in the form of optimized methods, I believe that Sista is well-positioned
for real-time since it can be used before deployment. In fact we should
emphasise this in the papers we write on Sista.
The solution of Eliot makes sense.
To write a paper about that I need benchs showing result on real time
applications.
So there's quite some work to do before.
Post by Eliot Miranda
Greetings,
Post by Clément Bera
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 (SMS ok)
+ 1 415 287 3547 (no SMS)
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/617f0af6/attachment-0001.html>
Martin McClure
2014-09-19 22:11:24 UTC
Permalink
Post by phil
The book that explains the best how to implement a high performance VM
for Smalltalk and why is Urs Holzle phd
<http://www.cs.ucsb.edu/~urs/oocsb/self/papers/urs-thesis.html>.
Agreed. This is good (almost required) reading for anyone who wants to
understand how to implement dynamic languages in a way that is not slow,
and to understand why performance of dynamic languages does not need to
be much slower than that of statically-typed languages.

After reading this paper, it's also good to think about the fact that it
describes work that was done over 20 years ago, and that hardware has
changed a great deal in the interim, and think hard about what
improvements might be made today over the techniques that Urs and the
Self team came up with back then.

Regards,

-Martin
Thierry Goubier
2014-09-16 05:09:06 UTC
Permalink
Hi Ronie,
Post by Clément Bera
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
Lowcode is currently a spec of new bytecode instructions. These
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new
bytecode set for SistaV1, which is called "inline primitives". Because
of this, these new instructions can be mixed freely with the standard
VM bytecode set. This also allows the Sista adaptive optimizer to
inline FFI calls.
- Int32 and Int64 integer arithmetic without type checking.
Will you provide SSE-type vector arithmetic or will you rely on the
compiler to generate those?
Post by Clément Bera
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
Lot of very significant stuff :)
Post by Clément Bera
- A C interpreter plugin.
- A LLVM based backend.
Do you mean a LLVM-IR to Lowcode backend compiler ?
Post by Clément Bera
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions
calls and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is
missing a better Slang generator, that allows to force better inlining
and more code reviews. There is a lot of code duplication in Cog, that
can be attributed to limitations of Slang. In my opinion, if we could
use Slang not only for building the VM we should end with a better
code generator. In addition we, need more people working in Cog. We
need people that performs code reviews and documentation of Cog.
So there is a path there about improving Slang to C generation.
Post by Clément Bera
After these weeks, I learned that working in Cogit it is not that
hard. Our biggest problem is lack of documentation. Our second problem
could be the lack of documentation about Slang
---------------------------------------------------------------
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to
optimize things such as loops with lot of arithmetics. It is designed
to optimize large sections of code. In Smalltalk, most of our code is
composed mostly of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations,
such as constant folding, common subexpressions, dead branch
elimination, loop unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
Do gradual typing of performance sensitive parts of the code.
Post by Clément Bera
In other words, our best bet is in the work of Cl?ment in Sista. The
only problem with this bet are real time applications.
Real time applications requires an upper bound guarantee in their
response time. In some cases, the lack of this guarantee can be just
an annoyance, as happens in video games. In some mission critical
applications the results can not be good, if this time constraint is
not met. An example of a mission critical system could the flight
controls of an airplane, or the cooling system of a nuclear reactor.
Don't worry/don't bother with thoses: you will never use Smalltalk or a
VM :) It will never be certified by authorities, and the industry will
never accept it.

Anyway, they are not even talking of coding anymore these days in there.
They do MDE.

Consider that the target can be named performance critical applications,
and those are significant but have different approaches to performance:
use highly tuned libraries (Intel mkl);
perform source to source analysis to extract performance;
vectorize code (via gcc intrinsics, SSE inline, OpenCL, ICC);
use inline asm;
use totally non-obvious algorithms (bitonic sort, bit slicing);
use GPUs;
use massively parallel computing resources;
load pre-optimised code generators in your code (compilette: a
minimized Jit which generate asm at a cost of 3 inst for one generated)

Adaptative optimisation is a very impressive technique, but it goes only
that far. When you consider that some compilation phases may require a
64 CPU cores machine to shorten the compilation time, then you don't do
it on the fly.
Post by Clément Bera
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
And of course, you have to perform lot of profiling.
Allways!

Thierry
Post by Clément Bera
Greetings,
Ronie
2014-09-15 16:38 GMT-03:00 Craig Latta <craig at netjam.org
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org <http://netjam.org>
+31 6 2757 7177 <tel:%2B31%20%20%206%202757%207177> (SMS ok)
+ 1 415 287 3547 <tel:%2B%201%20415%20%20287%203547> (no SMS)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/7acf6984/attachment.html>
Ben Coman
2014-09-16 11:14:26 UTC
Permalink
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/411c4fe1/attachment.html>
Thierry Goubier
2014-09-16 11:48:56 UTC
Permalink
Don't worry/don't bother with thoses: you will never use Smalltalk or a VM
:) It will never be certified by authorities, and the industry will never
accept it.
You are probably right for those two examples, but there are other
not-so-regulated domains where real-time is useful - e.g. industrial
automation and robotics.
Real-time is usefull there, yes. But Smalltalk and Cog will never get
there. Except as a DSL / code generator tool (which means a MDE approach,
more or less).

(And code generation is where Pharo to C or LLVM-IR gets us interested)

Dynamic optimisations, lack of static typing: they will laugh you out in
any of those fields.

Even if their developpers use Python behind their back.

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/a1f99efc/attachment.html>
kilon alios
2014-09-16 12:19:57 UTC
Permalink
Python is a great language why one would use it behind anyone's back ?

Python has several ways to deal with static typing, performance and
optimization, cython, shedskin , pypy and lately its creators linked a new
kind of python that allows for static types in his twitter feed.

There are also ways for python to take advantage LLVM one such project is
by Dropbox people (Where the creators of Python currently works for) called
"pyston" for high performance python -->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/


and the list goes on.... and on..... and on....



On Tue, Sep 16, 2014 at 2:48 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by Thierry Goubier
Don't worry/don't bother with thoses: you will never use Smalltalk or a
VM :) It will never be certified by authorities, and the industry will
never accept it.
You are probably right for those two examples, but there are other
not-so-regulated domains where real-time is useful - e.g. industrial
automation and robotics.
Real-time is usefull there, yes. But Smalltalk and Cog will never get
there. Except as a DSL / code generator tool (which means a MDE approach,
more or less).
(And code generation is where Pharo to C or LLVM-IR gets us interested)
Dynamic optimisations, lack of static typing: they will laugh you out in
any of those fields.
Even if their developpers use Python behind their back.
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/1bd83f38/attachment.html>
Thierry Goubier
2014-09-16 12:33:12 UTC
Permalink
Post by kilon alios
Python is a great language why one would use it behind anyone's back ?
Python has several ways to deal with static typing, performance and
optimization, cython, shedskin , pypy and lately its creators linked a new
kind of python that allows for static types in his twitter feed.
There are also ways for python to take advantage LLVM one such project is
by Dropbox people (Where the creators of Python currently works for) called
"pyston" for high performance python -->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
and the list goes on.... and on..... and on....
Everything you describe are partial solutions which suits the Python
community well for their needs. If python was suited for RT, embedded work,
it would be an horror to work with. The same goes with Smalltalk.

You're not supposed to have fun when you do embedded, RT work. You are
supposed to suffer :( This is no place for the creative spirit.

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/810ae940/attachment.html>
Serge Stinckwich
2014-09-16 12:55:02 UTC
Permalink
On Tue, Sep 16, 2014 at 2:33 PM, Thierry Goubier
Post by Thierry Goubier
Post by kilon alios
Python is a great language why one would use it behind anyone's back ?
Python has several ways to deal with static typing, performance and
optimization, cython, shedskin , pypy and lately its creators linked a new
kind of python that allows for static types in his twitter feed.
There are also ways for python to take advantage LLVM one such project is
by Dropbox people (Where the creators of Python currently works for) called
"pyston" for high performance python -->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
and the list goes on.... and on..... and on....
Everything you describe are partial solutions which suits the Python
community well for their needs. If python was suited for RT, embedded work,
it would be an horror to work with. The same goes with Smalltalk.
You're not supposed to have fun when you do embedded, RT work. You are
supposed to suffer :( This is no place for the creative spirit.
You are old fashion Thierry, now dynamic languages are used for
embedded stuff :-)
- Ruby to control robots: http://artoo.io/
- Javascript for drone : http://nodebots.io/
- Javascript for embedded boards: https://tessel.io/
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/
Thierry Goubier
2014-09-16 13:15:50 UTC
Permalink
Post by Serge Stinckwich
On Tue, Sep 16, 2014 at 2:33 PM, Thierry Goubier
Post by Thierry Goubier
Post by kilon alios
Python is a great language why one would use it behind anyone's back ?
Python has several ways to deal with static typing, performance and
optimization, cython, shedskin , pypy and lately its creators linked a
new
Post by Thierry Goubier
Post by kilon alios
kind of python that allows for static types in his twitter feed.
There are also ways for python to take advantage LLVM one such project
is
Post by Thierry Goubier
Post by kilon alios
by Dropbox people (Where the creators of Python currently works for)
called
Post by Thierry Goubier
Post by kilon alios
"pyston" for high performance python -->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
Post by Thierry Goubier
Post by kilon alios
and the list goes on.... and on..... and on....
Everything you describe are partial solutions which suits the Python
community well for their needs. If python was suited for RT, embedded
work,
Post by Thierry Goubier
it would be an horror to work with. The same goes with Smalltalk.
You're not supposed to have fun when you do embedded, RT work. You are
supposed to suffer :( This is no place for the creative spirit.
You are old fashion Thierry, now dynamic languages are used for
embedded stuff :-)
- Ruby to control robots: http://artoo.io/
- Javascript for drone : http://nodebots.io/
- Javascript for embedded boards: https://tessel.io/
Ah, Serge, if those could be relevant ;)

Only dreamers believe one can do good software with dynamic, creative
approaches. Of course they may succeed at one point, because there is no
proof we need to put a straightjacket to a developper to get safe, fast and
good quality code.

Nowadays, you can't even do programming language research without doing a
type system, and proofs and .... FP and static type systems have won, and
there is nothing left of the creative spirit in programming.

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/a94388c1/attachment.html>
kilon alios
2014-09-16 12:59:51 UTC
Permalink
I don't know maybe you are correct I have not coded for embeded platforms
so I dont offer opinion about things I dont know. But I do know is that
once the creator of python himself claimed that if you use python for
performance then you are using the wrong language. But hey what does he
know , he only created python . People having proving him so wrong for
years now with many libraries targeting top performance. What makes those
people special is what makes all innovators special, their unwillingness to
compromise, to accept defeat and stop asking "What if?".

So I am sorry I cant digest with "you are not supposed to have fun
with...." I am sure you are correct and you speak from experience but I
"just dont have the stomach to digest it" as we say here in Greece

PS: What is RT ?

On Tue, Sep 16, 2014 at 3:33 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by kilon alios
Python is a great language why one would use it behind anyone's back ?
Python has several ways to deal with static typing, performance and
optimization, cython, shedskin , pypy and lately its creators linked a new
kind of python that allows for static types in his twitter feed.
There are also ways for python to take advantage LLVM one such project is
by Dropbox people (Where the creators of Python currently works for) called
"pyston" for high performance python -->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
and the list goes on.... and on..... and on....
Everything you describe are partial solutions which suits the Python
community well for their needs. If python was suited for RT, embedded work,
it would be an horror to work with. The same goes with Smalltalk.
You're not supposed to have fun when you do embedded, RT work. You are
supposed to suffer :( This is no place for the creative spirit.
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/ac80f356/attachment.html>
Thierry Goubier
2014-09-16 13:23:55 UTC
Permalink
Post by kilon alios
I don't know maybe you are correct I have not coded for embeded platforms
so I dont offer opinion about things I dont know. But I do know is that
once the creator of python himself claimed that if you use python for
performance then you are using the wrong language. But hey what does he
know , he only created python . People having proving him so wrong for
years now with many libraries targeting top performance. What makes those
people special is what makes all innovators special, their unwillingness to
compromise, to accept defeat and stop asking "What if?".
There is a body of work on making Python fast, because Python is so cool
that people keep using it against all odds :) I'm working on making R fast,
for the same reason: non programmers using it to do big data.

Think of using Python + numPy on a 100 000 CPUs supercomputer :) Fun !

Well, of course, you have scores of project trying to show that writing the
same code in C++ (with the right DSL or framework) would be a hell of a lot
faster and safer because of static types.

Safer because non programmers wouldn't be writing code anymore, yes!
Post by kilon alios
So I am sorry I cant digest with "you are not supposed to have fun
with...." I am sure you are correct and you speak from experience but I
"just dont have the stomach to digest it" as we say here in Greece
Sorry, but this is real. Tales of the industry... It's hard to stomach when
you talk Research directions with those guys :(
Post by kilon alios
PS: What is RT ?
Real-Time.

Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/4d3df1c2/attachment-0001.html>
kilon alios
2014-09-16 13:34:58 UTC
Permalink
so its a people problem more than a technology problem from what you saying
. As it is in most cases.

I don't care if its python but 100k cpus sounds a lot of fun. Huge
potential :)

"Safer because non programmers wouldn't be writing code anymore, yes!"

if you write programs you are a programmer , no ?

Can you ever be safe in this life ? I think not . I don't believe in
holy grail and static types look like a holy grail to me, similar to
functional coding style. I can understand the benefit cant understand all
this hype. But then I am not too attached to dynamic typing as well. I just
dont want complexity imposed on me that I dont need, I want to be the boss
of my code and no stupid language or language designer tell me how to code
as if he know the needs of millions different coders. There I said it :D

On Tue, Sep 16, 2014 at 4:23 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by kilon alios
I don't know maybe you are correct I have not coded for embeded platforms
so I dont offer opinion about things I dont know. But I do know is that
once the creator of python himself claimed that if you use python for
performance then you are using the wrong language. But hey what does he
know , he only created python . People having proving him so wrong for
years now with many libraries targeting top performance. What makes those
people special is what makes all innovators special, their unwillingness to
compromise, to accept defeat and stop asking "What if?".
There is a body of work on making Python fast, because Python is so cool
that people keep using it against all odds :) I'm working on making R fast,
for the same reason: non programmers using it to do big data.
Think of using Python + numPy on a 100 000 CPUs supercomputer :) Fun !
Well, of course, you have scores of project trying to show that writing
the same code in C++ (with the right DSL or framework) would be a hell of a
lot faster and safer because of static types.
Safer because non programmers wouldn't be writing code anymore, yes!
Post by kilon alios
So I am sorry I cant digest with "you are not supposed to have fun
with...." I am sure you are correct and you speak from experience but I
"just dont have the stomach to digest it" as we say here in Greece
Sorry, but this is real. Tales of the industry... It's hard to stomach
when you talk Research directions with those guys :(
Post by kilon alios
PS: What is RT ?
Real-Time.
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/402d9bf1/attachment.html>
Eliot Miranda
2014-09-16 22:27:34 UTC
Permalink
On Tue, Sep 16, 2014 at 5:33 AM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by kilon alios
Python is a great language why one would use it behind anyone's back ?
Python has several ways to deal with static typing, performance and
optimization, cython, shedskin , pypy and lately its creators linked a new
kind of python that allows for static types in his twitter feed.
There are also ways for python to take advantage LLVM one such project is
by Dropbox people (Where the creators of Python currently works for) called
"pyston" for high performance python -->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
and the list goes on.... and on..... and on....
Everything you describe are partial solutions which suits the Python
community well for their needs. If python was suited for RT, embedded work,
it would be an horror to work with. The same goes with Smalltalk.
You're not supposed to have fun when you do embedded, RT work. You are
supposed to suffer :( This is no place for the creative spirit.
I think you need therapy ;-) Why not be positive and think about how you
could make these technologies work? I'm sure Smalltalk and Cog can do soft
real-time. People have been using it to do music for as long as its been
in existence. If you could get beyond your "it can't be done" mind set and
get into a "how might this be done" mindset you might make a real
contribution. Right now you're being a real downer.
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/15e16295/attachment.html>
Thierry Goubier
2014-09-17 17:13:02 UTC
Permalink
Post by Eliot Miranda
On Tue, Sep 16, 2014 at 5:33 AM, Thierry Goubier
2014-09-16 14:19 GMT+02:00 kilon alios <kilon.alios at gmail.com
Python is a great language why one would use it behind anyone's back ?
Python has several ways to deal with static typing, performance
and optimization, cython, shedskin , pypy and lately its
creators linked a new kind of python that allows for static
types in his twitter feed.
There are also ways for python to take advantage LLVM one such
project is by Dropbox people (Where the creators of Python
currently works for) called "pyston" for high performance python
-->
https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-python-implementation/
and the list goes on.... and on..... and on....
Everything you describe are partial solutions which suits the Python
community well for their needs. If python was suited for RT,
embedded work, it would be an horror to work with. The same goes
with Smalltalk.
You're not supposed to have fun when you do embedded, RT work. You
are supposed to suffer :( This is no place for the creative spirit.
I think you need therapy ;-) Why not be positive and think about how
you could make these technologies work? I'm sure Smalltalk and Cog can
do soft real-time. People have been using it to do music for as long as
its been in existence. If you could get beyond your "it can't be done"
mind set and get into a "how might this be done" mindset you might make
a real contribution. Right now you're being a real downer.
Been there, done that. You do it once, after you dump the people in that
field and you do more valuable stuff :)

I really needed that Smalltalk therapy ;)

Thierry
phil
2014-09-16 13:03:12 UTC
Permalink
On Tue, Sep 16, 2014 at 1:48 PM, Thierry Goubier <thierry.goubier at gmail.com>
Post by Thierry Goubier
Post by Thierry Goubier
Don't worry/don't bother with thoses: you will never use Smalltalk or a
VM :) It will never be certified by authorities, and the industry will
never accept it.
Post by Thierry Goubier
Post by Thierry Goubier
You are probably right for those two examples, but there are other
not-so-regulated domains where real-time is useful - e.g. industrial
automation and robotics.
Post by Thierry Goubier
Real-time is usefull there, yes. But Smalltalk and Cog will never get
there. Except as a DSL / code generator tool (which means a MDE approach,
more or less).
Post by Thierry Goubier
(And code generation is where Pharo to C or LLVM-IR gets us interested)
Dynamic optimisations, lack of static typing: they will laugh you out in
any of those fields.
Post by Thierry Goubier
Even if their developpers use Python behind their back.
http://www.altreonic.com/ has OpenComRTOSwhich looks the kind of thing we
would like to target then.

I know the creator of OpenCOMRTOS, in fact, he lives close to my place.

http://www.altreonic.com/sites/default/files/Altreonic%20OpenComRTOS2013.pdf

They happen to have a "VM"
http://www.altreonic.com/sites/default/files/Altreonic%20Safe%20Virtual%20Machine_C_2013.pdf


"The ultra small target independent Virtual Machine"

Applications:
? Remote diagnostics.
? Fail safe and fault tolerant control.
? Processor independent programming.
Thanks to the use of OpenComRTOS, SafeVM tasks can operate system wide
across
all nodes in the network. The user can also put several SafeVM tasks on the
same
node. The natively running OpenComRTOS itself acts as a virtual machine for
the SVM
tasks, isolating them from the underlying hardware details while providing
full access.
Safe Virtual Machine for C

So, looks like we aren't in such a black and white situation.

Phil
Post by Thierry Goubier
Thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/5d2fa608/attachment.html>
Eliot Miranda
2014-09-16 22:13:56 UTC
Permalink
Post by Eliot Miranda
Hi Ronie,
Hello,
I am segmenting this mail in several sections.
---------------------------------------------------------------
- On Lowcode and Cog
I have been working in the last week with the Cog VM, implementing the
Lowcode instructions in Cog.
Lowcode is currently a spec of new bytecode instructions. These
- Implementing a C like language compiler.
- Making FFI calls
I am implementing these instructions using a feature of the new bytecode
set for SistaV1, which is called "inline primitives". Because of this,
these new instructions can be mixed freely with the standard VM bytecode
set. This also allows the Sista adaptive optimizer to inline FFI calls.
- Int32 and Int64 integer arithmetic without type checking.
Will you provide SSE-type vector arithmetic or will you rely on the
compiler to generate those?
- Pointers, with arithmetics.
- Memory access and memory manipulation.
- Single and double precision floating point arithmetics.
- Conversion between primitive types.
- Boxing and unboxing of primitive types.
- Unchecked comparisons.
- Native function call. Direct and indirect calls.
- The atomic operation compare and swap.
- Object pin/unpin (requires Spur).
- VM releasing and grabbing for threaded ffi.
Lot of very significant stuff :)
- A C interpreter plugin.
- A LLVM based backend.
Do you mean a LLVM-IR to Lowcode backend compiler ?
Currently I am working in getting this working using the Cog code
generator. So far I am already generating code for
int32/pointer/float32/float64. I am starting to generate C functions calls
and object boxing/unboxing.
During this work I learned a lot about Cog. Specially that Cog is
missing a better Slang generator, that allows to force better inlining and
more code reviews. There is a lot of code duplication in Cog, that can be
attributed to limitations of Slang. In my opinion, if we could use Slang
not only for building the VM we should end with a better code generator. In
addition we, need more people working in Cog. We need people that performs
code reviews and documentation of Cog.
So there is a path there about improving Slang to C generation.
After these weeks, I learned that working in Cogit it is not that hard.
Our biggest problem is lack of documentation. Our second problem could be
the lack of documentation about Slang
---------------------------------------------------------------
- Smalltalk -> LLVM ?
As for having a Smalltalk -> LLVM code generator. The truth is that we
will not gain anything. LLVM is a C compiler, which is designed to optimize
things such as loops with lot of arithmetics. It is designed to optimize
large sections of code. In Smalltalk, most of our code is composed mostly
of message sends. LLVM cannot optimize a message send.
To optimize a message send, you have to determine which is the method
that is going to respond to the message. Then you have to inline the
method. And then you can start performing the actual optimizations, such as
constant folding, common subexpressions, dead branch elimination, loop
unrolling, and a long etc.
Because we don't have information in the actual language (e.g. static
types a la C/C++/Java/C#) that tells us what is going to be the actual
method invoked by a message send, we have the following alternatives to
- Don't optimize anything.
- Perform a costly static global analysis of the whole program.
- Measure in runtime and hope for the best.
- Extend the language.
Do gradual typing of performance sensitive parts of the code.
In other words, our best bet is in the work of Cl?ment in Sista. The
only problem with this bet are real time applications.
Real time applications requires an upper bound guarantee in their
response time. In some cases, the lack of this guarantee can be just an
annoyance, as happens in video games. In some mission critical applications
the results can not be good, if this time constraint is not met. An example
of a mission critical system could the flight controls of an airplane, or
the cooling system of a nuclear reactor.
Don't worry/don't bother with thoses: you will never use Smalltalk or a VM
:) It will never be certified by authorities, and the industry will never
accept it.
You are probably right for those two examples, but there are other
not-so-regulated domains where real-time is useful - e.g. industrial
automation and robotics.
+100. And Smalltalk is being used in real-time applications already, e.g.
semiconductor fab machines. It used to be in all of Tektronix's
oscilloscaopes.
Post by Eliot Miranda
Anyway, they are not even talking of coding anymore these days in there.
They do MDE.
Consider that the target can be named performance critical applications,
use highly tuned libraries (Intel mkl);
perform source to source analysis to extract performance;
vectorize code (via gcc intrinsics, SSE inline, OpenCL, ICC);
use inline asm;
use totally non-obvious algorithms (bitonic sort, bit slicing);
use GPUs;
use massively parallel computing resources;
load pre-optimised code generators in your code (compilette: a
minimized Jit which generate asm at a cost of 3 inst for one generated)
Adaptative optimisation is a very impressive technique, but it goes only
that far. When you consider that some compilation phases may require a 64
CPU cores machine to shorten the compilation time, then you don't do it on
the fly.
For these application, it is not possible to rely in an adaptive
optimizer that can be triggered sometimes. In these application you have to
- Extend the language to hand optimize some performance critical
sections of code.
- Use another language to optimize these critical section.
- Use another language for the whole project.
And of course, you have to perform lot of profiling.
Allways!
Thierry
Greetings,
Ronie
Hear hear!
-C
[1] http://tinyurl.com/m66fx8y (original message)
--
Craig Latta
netjam.org
+31 6 2757 7177 <%2B31%20%20%206%202757%207177> (SMS ok)
+ 1 415 287 3547 <%2B%201%20415%20%20287%203547> (no SMS)
--
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/07a712a1/attachment.html>
Thierry Goubier
2014-09-17 17:01:31 UTC
Permalink
Post by Eliot Miranda
+100. And Smalltalk is being used in real-time applications already,
e.g. semiconductor fab machines. It used to be in all of Tektronix's
oscilloscaopes.
And I remember a HP network analyser whose GUI was written with Digitalk/V.

But this is work where you go dual: you put your real-time control loop
on a core, making sure it will never be interrupted, and you let the
soft stuff (GUI and all) outside of the RT loop. There I can see Cog and
Smalltalk, not in the inner loop.

Thierry
Alain Rastoul
2014-09-20 00:10:48 UTC
Permalink
Post by Eliot Miranda
I find this whole discussion depressing. It seems people would rather
put their energy in chasing quick fixes or other technologies instead of
contributing to the work that is being done in the existing VM. People
discuss using LLVM as if the code generation capabilities inside Cog
were somehow poor or have no chance of competing. Spur is around twice
as fast as the current memory manager, has much better support for the
FFI. Cl?ment and I, now with help from Ronie, are making excellent
progress towards an adaptive optimizer/speculative inliner that will
give us similar performance to V8 (the Google JavaScript VM, lead by
Lars Bak, who implemented the HotSpot VM (Smalltalk and Java)) et al.
We are trying to get person-power for a high-quality FFI and have a
prototype for a non-blocking VM. When we succeed C won't be any better
and so it won't be an interesting target. One will be able to program
entirely in Smalltalk and get excellent performance. But we need
effort. Collaboration.
Hi Eliot,

Not everybody has the necessary skills to help and contribute to your
work, my assembly skills are really faraway and outdated now (... little
frustration here :( ... )
but imho your work is unvaluable to pharo and smalltalk community
- just to mention it, I noticed a 30 to 50% gain in a small bench I
wrote for fun recently (a very dumb chess pawn moves generator) with the
last Spur vm
I was shocked :)
64bits + x2 perfs + non blocking (or multi threaded?) vm are giant steps
forward
that makes it possible for pharo smalltalk to compete with mainstream
technologies

Regards,

Alain
phil
2014-09-20 00:33:25 UTC
Permalink
Post by Thierry Goubier
Post by Eliot Miranda
I find this whole discussion depressing. It seems people would rather
put their energy in chasing quick fixes or other technologies instead of
contributing to the work that is being done in the existing VM. People
discuss using LLVM as if the code generation capabilities inside Cog
were somehow poor or have no chance of competing. Spur is around twice
as fast as the current memory manager, has much better support for the
FFI. Cl?ment and I, now with help from Ronie, are making excellent
progress towards an adaptive optimizer/speculative inliner that will
give us similar performance to V8 (the Google JavaScript VM, lead by
Lars Bak, who implemented the HotSpot VM (Smalltalk and Java)) et al.
We are trying to get person-power for a high-quality FFI and have a
prototype for a non-blocking VM. When we succeed C won't be any better
and so it won't be an interesting target. One will be able to program
entirely in Smalltalk and get excellent performance. But we need
effort. Collaboration.
Hi Eliot,
Not everybody has the necessary skills to help and contribute to your
work, my assembly skills are really faraway and outdated now (... little
frustration here :( ... )
Post by Thierry Goubier
but imho your work is unvaluable to pharo and smalltalk community
for fun recently (a very dumb chess pawn moves generator) with the last
Spur vm
Post by Thierry Goubier
I was shocked :)
64bits + x2 perfs + non blocking (or multi threaded?) vm are giant steps
forward
Post by Thierry Goubier
that makes it possible for pharo smalltalk to compete with mainstream
technologies

We'll prevail!

What we also need to do is to showcase the Smalltalk workflow and how nice
it is.

We cannot gain mindshare by technical prowess only.

Phil
Post by Thierry Goubier
Regards,
Alain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140920/adebb796/attachment-0001.html>
Alain Rastoul
2014-09-20 01:26:46 UTC
Permalink
Post by phil
We'll prevail!
What we also need to do is to showcase the Smalltalk workflow and how
nice it is.
We cannot gain mindshare by technical prowess only.
Phil
I totally agree with you,
but I think that those technical advances are vital in current
technology trends (saas, cloud, server side dev) and progress in this
field is important.

Despite the fact I think that Smalltalk still has an advance in object
oriented systems/development, and offers a different perception it is
hard to get the point about object system with people who are formatted
by the hype or by their experience. Discussing with collegues about some
smalltalk concepts is always interesting :)
phil
2014-09-20 07:45:24 UTC
Permalink
A
Post by Alain Rastoul
Post by phil
We'll prevail!
What we also need to do is to showcase the Smalltalk workflow and how
nice it is.
We cannot gain mindshare by technical prowess only.
Phil
I totally agree with you,
but I think that those technical advances are vital in current
technology trends (saas, cloud, server side dev) and progress in this field
is important.
Post by Alain Rastoul
Despite the fact I think that Smalltalk still has an advance in object
oriented systems/development, and offers a different perception it is hard
to get the point about object system with people who are formatted by the
hype or by their experience. Discussing with collegues about some smalltalk
concepts is always interesting :)
I am facing quite some people who are in the FP bandwagon and wonder how to
bridge with their views. Like classes are just functions etc.

Phil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140920/822c015a/attachment.html>
Frank Shearar
2014-09-20 08:56:53 UTC
Permalink
Post by phil
A
Post by Alain Rastoul
Post by phil
We'll prevail!
What we also need to do is to showcase the Smalltalk workflow and how
nice it is.
We cannot gain mindshare by technical prowess only.
Phil
I totally agree with you,
but I think that those technical advances are vital in current technology
trends (saas, cloud, server side dev) and progress in this field is
important.
Despite the fact I think that Smalltalk still has an advance in object
oriented systems/development, and offers a different perception it is hard
to get the point about object system with people who are formatted by the
hype or by their experience. Discussing with collegues about some smalltalk
concepts is always interesting :)
I am facing quite some people who are in the FP bandwagon and wonder how to
bridge with their views. Like classes are just functions etc.
The article that convinced me that objects are higher order functions
(that close over a set of variables) is here:
http://letoverlambda.com/index.cl/guest/chap2.html

The meat of the article (as far as class/object/function
correspondences goes) starts at "Let over Lambda".

frank
Post by phil
Phil
phil
2014-09-20 10:04:13 UTC
Permalink
Post by Frank Shearar
Post by phil
A
Post by Alain Rastoul
Post by phil
We'll prevail!
What we also need to do is to showcase the Smalltalk workflow and how
nice it is.
We cannot gain mindshare by technical prowess only.
Phil
I totally agree with you,
but I think that those technical advances are vital in current technology
trends (saas, cloud, server side dev) and progress in this field is
important.
Despite the fact I think that Smalltalk still has an advance in object
oriented systems/development, and offers a different perception it is hard
to get the point about object system with people who are formatted by the
hype or by their experience. Discussing with collegues about some smalltalk
concepts is always interesting :)
I am facing quite some people who are in the FP bandwagon and wonder how to
bridge with their views. Like classes are just functions etc.
The article that convinced me that objects are higher order functions
http://letoverlambda.com/index.cl/guest/chap2.html
The meat of the article (as far as class/object/function
correspondences goes) starts at "Let over Lambda".
Thanks for the reference.

I still can't get myself to plunge into that I've to admit.

The issue is that there are not too many people grasping this in typical
business applications. Not to mention staffing teams.

How do you use this in practice?

I know you do lots of magic at your company. I also know that therr aren't
that many Frank S. around...

Phil
Post by Frank Shearar
frank
Post by phil
Phil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140920/2f3cb016/attachment.html>
Frank Shearar
2014-09-21 15:07:47 UTC
Permalink
Post by phil
Post by Frank Shearar
Post by phil
A
Post by Alain Rastoul
Post by phil
We'll prevail!
What we also need to do is to showcase the Smalltalk workflow and how
nice it is.
We cannot gain mindshare by technical prowess only.
Phil
I totally agree with you,
but I think that those technical advances are vital in current technology
trends (saas, cloud, server side dev) and progress in this field is
important.
Despite the fact I think that Smalltalk still has an advance in object
oriented systems/development, and offers a different perception it is hard
to get the point about object system with people who are formatted by the
hype or by their experience. Discussing with collegues about some smalltalk
concepts is always interesting :)
I am facing quite some people who are in the FP bandwagon and wonder how to
bridge with their views. Like classes are just functions etc.
The article that convinced me that objects are higher order functions
http://letoverlambda.com/index.cl/guest/chap2.html
The meat of the article (as far as class/object/function
correspondences goes) starts at "Let over Lambda".
Thanks for the reference.
I still can't get myself to plunge into that I've to admit.
It certainly helps if you can speak a bit of Lisp. Or an ML language,
if you just ignore the ()s. "let over fun" would be the ML/Haskell/F#
version.
Post by phil
The issue is that there are not too many people grasping this in typical
business applications. Not to mention staffing teams.
How do you use this in practice?
I don't :) Mostly, I find this stuff valuable in understanding new
programming languages. If you know the bones of computing, you
recognise the common patterns between languages. Helps me get up to
speed with new technologies.
Post by phil
I know you do lots of magic at your company. I also know that therr aren't
that many Frank S. around...
Sadly, I don't do much magic at work anymore: just doing CRUD in the
cloud nowadays, and trying to talk sense into my colleagues.

frank
Post by phil
Phil
Post by Frank Shearar
frank
Post by phil
Phil
phil
2014-09-21 15:28:51 UTC
Permalink
I may have more interesting fish to fry on the horizon...

Let's keep in touch :-)

Phil
Post by Eliot Miranda
Le 20 sept. 2014 10:57, "Frank Shearar" <frank.shearar at gmail.com> a
On 20 September 2014 08:45, phil at highoctane.be <phil at highoctane.be>
A
Le 20 sept. 2014 03:27, "Alain Rastoul" <alf.mmm.cat at gmail.com> a
?crit
Post by Alain Rastoul
Post by phil
We'll prevail!
What we also need to do is to showcase the Smalltalk workflow and
how
Post by Alain Rastoul
Post by phil
nice it is.
We cannot gain mindshare by technical prowess only.
Phil
I totally agree with you,
but I think that those technical advances are vital in current technology
trends (saas, cloud, server side dev) and progress in this field is
important.
Despite the fact I think that Smalltalk still has an advance in
object
Post by Alain Rastoul
oriented systems/development, and offers a different perception it is hard
to get the point about object system with people who are formatted by the
hype or by their experience. Discussing with collegues about some smalltalk
concepts is always interesting :)
I am facing quite some people who are in the FP bandwagon and wonder
how
to
bridge with their views. Like classes are just functions etc.
The article that convinced me that objects are higher order functions
http://letoverlambda.com/index.cl/guest/chap2.html
The meat of the article (as far as class/object/function
correspondences goes) starts at "Let over Lambda".
Thanks for the reference.
I still can't get myself to plunge into that I've to admit.
It certainly helps if you can speak a bit of Lisp. Or an ML language,
if you just ignore the ()s. "let over fun" would be the ML/Haskell/F#
version.
The issue is that there are not too many people grasping this in typical
business applications. Not to mention staffing teams.
How do you use this in practice?
I don't :) Mostly, I find this stuff valuable in understanding new
programming languages. If you know the bones of computing, you
recognise the common patterns between languages. Helps me get up to
speed with new technologies.
I know you do lots of magic at your company. I also know that therr
aren't
that many Frank S. around...
Sadly, I don't do much magic at work anymore: just doing CRUD in the
cloud nowadays, and trying to talk sense into my colleagues.
frank
Phil
frank
Phil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140921/25c7fe27/attachment.html>
Alain Rastoul
2014-09-20 11:02:21 UTC
Permalink
Post by Frank Shearar
Post by phil
I am facing quite some people who are in the FP bandwagon and wonder how to
bridge with their views. Like classes are just functions etc.
The article that convinced me that objects are higher order functions
http://letoverlambda.com/index.cl/guest/chap2.html
The meat of the article (as far as class/object/function
correspondences goes) starts at "Let over Lambda".
frank
Post by phil
Phil
Woaw, very interesting, thank you for sharing :)

Coding in dotnet at work, I sometime think about smalltalk objects
methods as a set of lambda within a scope : the "object".
To me, lambda use in c# lacks of organization and somehow leads to
anarchy, functional programming fills this gap with a different way of
thinking (I had fun doing some f# workshops at ms tech days).
Your high order assumption is intriguing, I have to figure it.

I often wondered myself how to visualize methods and message flow in
Roassal (I feel that perception and representation is a key here)

Regards,
Alain
phil
2014-09-15 13:14:39 UTC
Permalink
Sure, but enough of Smalltalk flavor to not want to go into C.
For specific plugins, this is really cool to have.

Phil


On Mon, Sep 15, 2014 at 2:39 PM, Cl?ment Bera <bera.clement at gmail.com>
Post by Clément Bera
Hello,
Note that slang is a subset of smalltalk. The Slang compiler does not
allow to compile smalltalk to C. It allows to compile a smalltalk with
restricted message sends and classes to C.
Post by Thierry Goubier
Hi Phil,
thanks for the update on Slang to C. Allways significant to have that.
- would a slang to x86 asm via NativeBoost be doable / a nice target?
Yes it would be interesting. However, by having a Slang to C compiler,
we're platform-independent, we can compile the C code to x86, x86_64 and
ARM quite easily (some part of the VM are already processor dependent, but
not so much). Targeting direct machine code implies evolving the Slang
compiler for each new assembly code we support. It sounds like a lot of
engineering work compared to our resources and the gain.
Post by Thierry Goubier
- would targetting LLVM-IR be of interest?
If you compile the C code with Clang instead of gcc, which starts to be
the case because of the lack of support for gcc in the latest Mac OS X, you
are already using LLVM IR because Clang uses it. As the VM use the GNU C
extensions to improve performance, I do not think that targeting directly
the LLVM IR would greatly improve performance. So it sounds like quite some
engineering work for no gain.
However, I think Ronie was interested in doing such work. If he succeeds
and reports performance improvement, then we can consider using his
compiler to compile the VM.
Post by Thierry Goubier
Thierry
Slang has been externalized by Pavel. So, Smalltalk to C works.
Post by phil
Works nicely, even if there were a few glitches (like code generated
twice at one point).
Nothing unfixable, I got the beast working.
Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.
PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).
Phil
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is
the pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project
related with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/e5122e83/attachment.html>
Pavel Krivanek
2014-09-16 07:52:15 UTC
Permalink
Post by phil
Slang has been externalized by Pavel. So, Smalltalk to C works.
Works nicely, even if there were a few glitches (like code generated twice
at one point).
Nothing unfixable, I got the beast working.
Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.
PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).
Phil
I hope only super slow :-) I simply have no time for this project now.
Help is welcome.

-- Pavel
Post by phil
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
santiagobragagnolo at gmail.com
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
2014-09-15 11:04 GMT+02:00 kilon alios <kilon.alios at gmail.com
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/ed7ff3a1/attachment.html>
phil
2014-09-16 09:39:13 UTC
Permalink
On Tue, Sep 16, 2014 at 9:52 AM, Pavel Krivanek <pavel.krivanek at gmail.com>
Post by Pavel Krivanek
Post by phil
Slang has been externalized by Pavel. So, Smalltalk to C works.
Works nicely, even if there were a few glitches (like code generated
twice at one point).
Nothing unfixable, I got the beast working.
Allows for things like: write Slang, generate C, compile into DLL, load
DLL, run C code. All in a single shot.
PavelKrivanek/CCodeGenerator on SmalltalkHub (which looks like super
slow/zombified).
Phil
I hope only super slow :-) I simply have no time for this project now.
Help is welcome.
I was speaking about Smalltalkhub, not your project :-)
Phil
Post by Pavel Krivanek
-- Pavel
Post by phil
On Mon, Sep 15, 2014 at 11:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is
the pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140916/c1d1f519/attachment.html>
Juan
2014-09-15 12:29:54 UTC
Permalink
Hola Santiago


Como se llama tal proyecto, me podria ser util podria colaborar

saludos

Juan Marcelo


On Mon, Sep 15, 2014 at 6:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/141f43d2/attachment.html>
Santiago Bragagnolo
2014-09-15 12:33:30 UTC
Permalink
Que proyecto?? ?_?

el que pasa smalltalk a c? estan hablando de slang
Post by Juan
Hola Santiago
Como se llama tal proyecto, me podria ser util podria colaborar
saludos
Juan Marcelo
On Mon, Sep 15, 2014 at 6:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/6af3084e/attachment.html>
Juan
2014-09-15 12:46:11 UTC
Permalink
Arduino to c++ /c

Best
Jmdc
El 15/09/2014 09:33, "Santiago Bragagnolo" <santiagobragagnolo at gmail.com>
Post by Santiago Bragagnolo
Que proyecto?? ?_?
el que pasa smalltalk a c? estan hablando de slang
Post by Juan
Hola Santiago
Como se llama tal proyecto, me podria ser util podria colaborar
saludos
Juan Marcelo
On Mon, Sep 15, 2014 at 6:28 AM, Santiago Bragagnolo <
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is
the pseudo smalltalk used to develop the VM.
Also there is a project for generating C for arduino, (a project related
with EToys), but i am not sure about how complete is.
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit
parser or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/3800fc0c/attachment.html>
David T. Lewis
2014-09-17 01:38:39 UTC
Permalink
Post by Santiago Bragagnolo
I may be wrong, but I think the closest thing out there is Slang. Is the
pseudo smalltalk used to develop the VM.
A bit off topic, but just for clarification: "Slang" is not a language. It
is just a casual name to refer to the subset of Smalltalk that can be easily
translated to a language such as C. Some of this Smalltalk may be written
in awkward ways in order to make sure it can be translated to a lower level
language, but it is real Smalltalk nonetheless.

The VM is not written in Slang, it is written in Smalltalk. When people talk
about running the VM simulator, they are talking about running the VM in its
native Smalltalk implementation, as opposed to running the code that is more
commonly translated to C and linked to the platform support code.

There is no psuedo Smalltalk involved, it is the real thing.

Dave
Thierry Goubier
2014-09-15 09:33:40 UTC
Permalink
Hi Kilon,

you don't need any parser to do that, you need something to generate C /
C++ code :)

There are Smalltalk to C translators floating around, and I believe one of
them is in the Squeak / Pharo code base to generate the VM (or part of it).

If you want to do it yourself, you probably should: explore the Ring model
as a way to export the package / class structure in C++ (just generate text
as you go) and also make a pass method by method, asking for the RBAST of
the method, and a visitor to generate the C / C++ equivalent to each node
of the AST (and you'll find the Smalltalk AST very simple and nice :) ).

Thierry
Post by kilon alios
Is there a way to convert code from pharo to c or c++ ? Does pettit parser
or other parsers offer such support ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20140915/e7fc9d9b/attachment.html>
Andres Valloud
2014-09-19 00:05:13 UTC
Permalink
Side comments...

Some code just has to be written in C, otherwise you cannot rely on the
behavior being correct and ultimately that's a major show stopper for
anything you might want to call "production" code. Anything having to
do with calling operating system libraries is a typical example of this
phenomenon. So, in those cases, translating Smalltalk to C (or
interfacing to OS libraries with FFI interfaces) only gives you huge
maintenance headaches in the long run. The resulting distractions due
to bug reports will obliterate any engineering team's performance, and
it's a race that cannot be won as framed. POSIX is there for a reason,
let the C compiler do the job it's been doing for 40+ years.

But suppose one wants to write C code with Smalltalk syntax anyway.
Nevertheless, C99 and newer still apply and one must be aware of the
subject matter. The danger is that Smalltalk is so much more user
friendly than C, that once one starts writing C in Smalltalk one will
never look at the automatically generated C. But is the mechanically
generated C code valid? Because if not, then compiling such source code
can be no better than "garbage in, garbage out", and then you cannot
have production quality. Similar arguments apply to FFI interfaces.
Please excuse my ignorance, but I'd like to assume someone's looking
into this?

What seems to follow is that the architecture of the system should try
to minimize the dependencies on external libraries, precisely because
the dependencies cause exposure to foreign rules and regulations that
slow one down. Creating dependencies in a reasonable manner should be
straightforward _for system users_, though. Letting externals create
reliable dependencies on one's system should also be facilitated to
promote healthy modes of interaction with the community at large.

I feel one of our current challenges is to wholeheartedly admit there is
no royal road to inventing our future, then act accordingly.

Andres.
Continue reading on narkive:
Loading...