Discussion:
[Pharo-dev] Contributing to Pharo
David Allouche
2016-01-15 19:45:17 UTC
Permalink
Hello,

I believe the Contributing to Pharo page http://pharo.org/contribute-propose-fix <http://pharo.org/contribute-propose-fix>
is out of date.

I found a bug
https://pharo.fogbugz.com/f/cases/17396/MonticelloRepositoryBrowser-example-MNU <https://pharo.fogbugz.com/f/cases/17396/MonticelloRepositoryBrowser-example-MNU>

and a made a fix in in a clean image (50525), created by the newly fixed PharoLauncher.

Then I followed the instruction on the page.
Open the Monticello Browser.
Open the Slice Maker by clicking the "Slice" button.
In Slice Maker enter the Fogbugz issue number and "grab" the issue summary.
Select all dirty packages affected by your bug fix.
I have no idea how the "list of dirty packages" is generated, but
it shows a bunch of packages I have not touched
it does not show the package that contains my fix (Spec-Tools)



I looked in the world menu, and I found this menu item called "Komitter". That sounded like a good candidate.



Now, I guess I should select Pharo50Inbox as the Remote, write a commit message, click Ok and hope for the best.

Is that right?

I guess contributors are expected to use a recent image. So I think it would be good that someone updates the documentation. Or fix the slice maker. Or both.
stepharo
2016-01-15 20:01:44 UTC
Permalink
Post by David Allouche
Hello,
I believe the Contributing to Pharo page
http://pharo.org/contribute-propose-fix
is out of date.
Certainly. Time passes so fast.
Post by David Allouche
I found a bug
https://pharo.fogbugz.com/f/cases/17396/MonticelloRepositoryBrowser-example-MNU
and a made a fix in in a clean image (50525), created by the newly fixed PharoLauncher.
Then I followed the instruction on the page.
o Open the Monticello Browser.
o Open the Slice Maker by clicking the "Slice" button.
o In Slice Maker enter the Fogbugz issue number and "grab" the
issue summary.
o Select all dirty packages affected by your bug fix.
I have no idea how the "list of dirty packages" is generated, but
1. it shows a bunch of packages I have not touched
Sometimes MC is confused and some packages get dirty while they are not.
Post by David Allouche
1. it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
Post by David Allouche
I looked in the world menu, and I found this menu item called
"Komitter". That sounded like a good candidate.
Now, I guess I should select Pharo50Inbox as the Remote, write a
commit message, click Ok and hope for the best.
Sounds correct.
I'm not sure that I understand Komitter :).
Post by David Allouche
Is that right?
I guess contributors are expected to use a recent image.
Yes it is always better and also to only publish one change at a time
(which is annoying when you have sequences of changes but
nice to track down changes).
Post by David Allouche
So I think it would be good that someone updates the documentation. Or
fix the slice maker. Or both.
Can you reproduce the bug?

BTW it would be nice that you print and sign the pharo license, then you
send us a scanned or surface mail version
and we will archive it.

We ask that to every contributors.

Stef
David Allouche
2016-01-15 22:16:24 UTC
Permalink
Post by stepharo
Post by David Allouche
it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
I do not know how MC shows a package as dirty.

And anyway, I ended up fiddling my way into uploading the version to smalltalkhub (I think I just clicked Save in Monticello Browser). So it no longer shows as dirty. But apparently that did not provide the fogzbugz issue metadata… So it might be not dirty anymore. But then I have no idea what to do now to upload it correctly.
Post by stepharo
Post by David Allouche
So I think it would be good that someone updates the documentation. Or fix the slice maker. Or both.
Can you reproduce the bug?
I cannot not reproduce it. I have no idea at all how people actually manage to use Slice Maker in a recent image.

Have you tried with a recent
Post by stepharo
BTW it would be nice that you print and sign the pharo license, then you send us a scanned or surface mail version
and we will archive it.
Surface mail, is that a kind of new social network thing? I'm afraid I got a bit behind the times those past few years.
Post by stepharo
We ask that to every contributors.
No problem, I'll do that. Copyright paperwork is important.
Nicolai Hess
2016-01-15 22:23:26 UTC
Permalink
Post by David Allouche
1. it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
I do not know how MC shows a package as dirty.
This is how it should look in slice maker (left side) and monticello
browser (right side).
The " * " marks dirty packages.
Nicolai Hess
2016-01-15 22:24:32 UTC
Permalink
Post by Nicolai Hess
Post by David Allouche
1. it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
I do not know how MC shows a package as dirty.
This is how it should look in slice maker (left side) and monticello
browser (right side).
The " * " marks dirty packages.
Maybe, we should include some screenshots on the "Contribute to Pharo" page.
Ferlicot D. Cyril
2016-01-15 22:27:38 UTC
Permalink
Post by Nicolai Hess
Maybe, we should include some screenshots on the "Contribute to Pharo" page.
And maybe a screencast :)
--
Cyril Ferlicot

http://www.synectique.eu

165 Avenue Bretagne
Lille 59000 France
David Allouche
2016-01-15 22:33:56 UTC
Permalink
Post by David Allouche
Post by stepharo
Post by David Allouche
it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
I do not know how MC shows a package as dirty.
This is how it should look in slice maker (left side) and monticello browser (right side).
The " * " marks dirty packages.
Okay, got that.

So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello, unexpected contextual menu showing up when I do Cmd-T in the debugger, infinite recursions when trying to debug through the MonticelloRepositoryBrowser... All things that are so obvious that I have trouble believing they are actual bugs. Maybe there was something wrong in how my image was generated. I will try building another image.

I also wonder
 so this "list of dirty packages" is actually a list of every package where some of them are marked dirty? I mean, why is it not just the list of dirty package, as the name implies?
David Allouche
2016-01-15 22:48:17 UTC
Permalink
Post by David Allouche
So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello, unexpected contextual menu showing up when I do Cmd-T in the debugger, infinite recursions when trying to debug through the MonticelloRepositoryBrowser... All things that are so obvious that I have trouble believing they are actual bugs. Maybe there was something wrong in how my image was generated. I will try building another image.
Built a new latest image. Now I no longer have any syntax highlighting in the debuggers or class browser…

Now, people should use the latest image for dev. But apparently the latest image is more often broken than not. What is the trick?

Sorry for the unpleasant messages, but I am getting incredibly frustrated.
Nicolai Hess
2016-01-15 22:55:36 UTC
Permalink
Post by David Allouche
Post by David Allouche
So yes, it's broken, but maybe it's something else entirely, I get new
reproducible errors every few minutes: errors when activating pretty
printing in the diff viewer in Monticello, unexpected contextual menu
showing up when I do Cmd-T in the debugger, infinite recursions when trying
to debug through the MonticelloRepositoryBrowser... All things that are so
obvious that I have trouble believing they are actual bugs. Maybe there was
something wrong in how my image was generated. I will try building another
image.
Built a new latest image. Now I no longer have any syntax highlighting in
the debuggers or class browser

Now, people should use the latest image for dev. But apparently the latest
image is more often broken than not. What is the trick?
Ah, that's not true. I work this way every day. And yes it can happen that
the latest image is broken but, not that often.
Post by David Allouche
Sorry for the unpleasant messages, but I am getting incredibly frustrated.
I am now (at this moment) working with the image version 50528.
David Allouche
2016-01-15 22:58:57 UTC
Permalink
Built a new latest image. Now I no longer have any syntax highlighting in the debuggers or class browser

Now, people should use the latest image for dev. But apparently the latest image is more often broken than not. What is the trick?
Ah, that's not true. I work this way every day. And yes it can happen that the latest image is broken but, not that often.
Sorry for the unpleasant messages, but I am getting incredibly frustrated.
I am now (at this moment) working with the image version 50528.
Yes, that's the one where syntax highlighting appears broken.
David Allouche
2016-01-15 23:07:25 UTC
Permalink
Post by David Allouche
Post by Nicolai Hess
I am now (at this moment) working with the image version 50528.
Yes, that's the one where syntax highlighting appears broken.
Nevermind, it works… I must have imagined it did not, or a transient bout of colour-blindness…
Nicolai Hess
2016-01-15 23:14:25 UTC
Permalink
Post by David Allouche
Post by Nicolai Hess
I am now (at this moment) working with the image version 50528.
Yes, that's the one where syntax highlighting appears broken.
Nevermind, it works
 I must have imagined it did not, or a transient bout
of colour-blindness

We have some problems with the current syntax highlighting, somehow
somtimes the highlighting is slow or paused.
We did not found a solution yet :
16020
<https://pharo.fogbugz.com/f/cases/16020/Syntax-Highlighting-Rubric-First-shows-non-highlighted-text-then-color-Looks-slow>
Syntax Highlighting Rubric: First shows non-highlighted text, then color.
Looks slow
Nicolai Hess
2016-01-15 22:49:48 UTC
Permalink
Post by David Allouche
Post by David Allouche
1. it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
I do not know how MC shows a package as dirty.
This is how it should look in slice maker (left side) and monticello browser (right side).
The " * " marks dirty packages.
Okay, got that.
So yes, it's broken, but maybe it's something else entirely, I get new
reproducible errors every few minutes: errors when activating pretty
printing in the diff viewer in Monticello,
this is a known error from the "recent" change for rubric text component
Post by David Allouche
unexpected contextual menu showing up when I do Cmd-T in the debugger,
known ( from the recent changes for debugger :) )
Post by David Allouche
infinite recursions when trying to debug through the
MonticelloRepositoryBrowser...
I don't know this error. But I don't think "this"
MonticelloRepositoryBrowser is actually used.
Post by David Allouche
All things that are so obvious that I have trouble believing they are
actual bugs. Maybe there was something wrong in how my image was generated.
I will try building another image.
I also wonder
 so this "list of dirty packages" is actually a list of
every package where some of them are marked dirty? I mean, why is it not
just the list of dirty package, as the name implies?
David Allouche
2016-01-15 22:57:00 UTC
Permalink
Post by David Allouche
So yes, it's broken, but maybe it's something else entirely, I get new reproducible errors every few minutes: errors when activating pretty printing in the diff viewer in Monticello,
this is a known error from the "recent" change for rubric text component
unexpected contextual menu showing up when I do Cmd-T in the debugger,
known ( from the recent changes for debugger :) )
infinite recursions when trying to debug through the MonticelloRepositoryBrowser...
I don't know this error. But I don't think "this" MonticelloRepositoryBrowser is actually used.
I think that was actually a bug in the debugger, or inspector, not in MonticelloRepositoryBrowser.

And yes, this appear to be dead code. I found it before finding the actual browser, and thought it might be a good place to start exploring Monticello.

I tried the old debugger in my new latest image (in the hope that syntax highlighting would not be broken there), but I realised it advertised no keyboard shortcut for stepping through the code


Starting to wonder if I should go back to spending my evenings playing Elite Dangerous, and come back in a few weeks when stuff is hopefully less buggy everywhere :-(
Henrik Sperre Johansen
2016-01-15 23:15:47 UTC
Permalink
Or, try a stable release.
Whenever I try developing on the latest version, I end up getting disrupted
by bugs in the core tools 70-80% of the time.
For released versions, in 1.4 it was maybe 5%, in 2.0 40%, in 3.0 20%, in
4.0 10%
So at least it's been trending downwards, though I expect a totally new
debugger, image format, RBParser unification and new FFI in 5.0 will keep
causing problems for a quite a while.

Cheers,
Henry
Post by Nicolai Hess
Post by David Allouche
So yes, it's broken, but maybe it's something else entirely, I get new
reproducible errors every few minutes: errors when activating pretty
printing in the diff viewer in Monticello,
this is a known error from the "recent" change for rubric text component
Post by David Allouche
unexpected contextual menu showing up when I do Cmd-T in the debugger,
known ( from the recent changes for debugger :) )
Post by David Allouche
infinite recursions when trying to debug through the
MonticelloRepositoryBrowser...
I don't know this error. But I don't think "this"
MonticelloRepositoryBrowser is actually used.
I think that was actually a bug in the debugger, or inspector, not in
MonticelloRepositoryBrowser.
And yes, this appear to be dead code. I found it before finding the actual
browser, and thought it might be a good place to start exploring Monticello.
I tried the old debugger in my new latest image (in the hope that syntax
highlighting would not be broken there), but I realised it advertised no
keyboard shortcut for stepping through the code

Starting to wonder if I should go back to spending my evenings playing
Elite Dangerous, and come back in a few weeks when stuff is hopefully less
buggy everywhere :-(
stepharo
2016-01-16 11:08:09 UTC
Permalink
Post by Henrik Sperre Johansen
Or, try a stable release.
Whenever I try developing on the latest version, I end up getting
disrupted by bugs in the core tools 70-80% of the time.
For released versions, in 1.4 it was maybe 5%, in 2.0 40%, in 3.0 20%,
in 4.0 10%
So at least it's been trending downwards, though I expect a totally
new debugger, image format, RBParser unification and new FFI in 5.0
will keep causing problems for a quite a while.
Yes the point is really to understand that we changed a lot of things
because they had to.
So this is important to put bugs in perspective to changes.

Now we should improve the test coverage too
Stef
Ben Coman
2016-01-16 01:42:31 UTC
Permalink
Post by Nicolai Hess
Post by David Allouche
So yes, it's broken, but maybe it's something else entirely, I get new
reproducible errors every few minutes: errors when activating pretty
printing in the diff viewer in Monticello,
this is a known error from the "recent" change for rubric text component
Post by David Allouche
unexpected contextual menu showing up when I do Cmd-T in the debugger,
known ( from the recent changes for debugger :) )
Post by David Allouche
infinite recursions when trying to debug through the
MonticelloRepositoryBrowser...
I don't know this error. But I don't think "this"
MonticelloRepositoryBrowser is actually used.
I think that was actually a bug in the debugger, or inspector, not in
MonticelloRepositoryBrowser.
And yes, this appear to be dead code. I found it before finding the actual
browser, and thought it might be a good place to start exploring Monticello.
I tried the old debugger in my new latest image (in the hope that syntax
highlighting would not be broken there), but I realised it advertised no
keyboard shortcut for stepping through the code…
Starting to wonder if I should go back to spending my evenings playing Elite
Dangerous, and come back in a few weeks when stuff is hopefully less buggy
everywhere :-(
Generally I'm impressed at Pharo's stability given depth of the
surgery that is sometimes done on it - but after all it is bleeding
edge and sometimes things get in a bind. Since all the tools are
built and run inside the Image, sometimes its a bit like changing the
tyres on a moving car - but there is the advantage that we are
*always* eating our own dogfood.

So if occasionally if everything seems against you, after trying a
fresh Image as you already did, slide back to the last build that
worked for you (I don't think the issues you experience are Spur
related, but 50496 might be a good fallback as Pre-Spur and
Pre-GTDebugger). Then create a fresh Image from the latest build to
test your Slice and in that time maybe the other issues have gone.

cheers -ben
Ben Coman
2016-01-16 02:30:43 UTC
Permalink
Post by Ben Coman
Post by Nicolai Hess
Post by David Allouche
So yes, it's broken, but maybe it's something else entirely, I get new
reproducible errors every few minutes: errors when activating pretty
printing in the diff viewer in Monticello,
this is a known error from the "recent" change for rubric text component
Post by David Allouche
unexpected contextual menu showing up when I do Cmd-T in the debugger,
known ( from the recent changes for debugger :) )
Post by David Allouche
infinite recursions when trying to debug through the
MonticelloRepositoryBrowser...
I don't know this error. But I don't think "this"
MonticelloRepositoryBrowser is actually used.
I think that was actually a bug in the debugger, or inspector, not in
MonticelloRepositoryBrowser.
And yes, this appear to be dead code. I found it before finding the actual
browser, and thought it might be a good place to start exploring Monticello.
I tried the old debugger in my new latest image (in the hope that syntax
highlighting would not be broken there), but I realised it advertised no
keyboard shortcut for stepping through the code…
Starting to wonder if I should go back to spending my evenings playing Elite
Dangerous, and come back in a few weeks when stuff is hopefully less buggy
everywhere :-(
Generally I'm impressed at Pharo's stability given depth of the
surgery that is sometimes done on it - but after all it is bleeding
edge and sometimes things get in a bind. Since all the tools are
built and run inside the Image, sometimes its a bit like changing the
tyres on a moving car - but there is the advantage that we are
*always* eating our own dogfood.
So if occasionally if everything seems against you, after trying a
fresh Image as you already did, slide back to the last build that
worked for you (I don't think the issues you experience are Spur
related, but 50496 might be a good fallback as Pre-Spur and
Whoops. That should be 50495.
Post by Ben Coman
Pre-GTDebugger). Then create a fresh Image from the latest build to
test your Slice and in that time maybe the other issues have gone.
cheers -ben
David Allouche
2016-01-16 09:07:34 UTC
Permalink
Post by Ben Coman
So if occasionally if everything seems against you, after trying a
fresh Image as you already did, slide back to the last build that
worked for you (I don't think the issues you experience are Spur
related, but 50496 might be a good fallback as Pre-Spur and
Pre-GTDebugger). Then create a fresh Image from the latest build to
test your Slice and in that time maybe the other issues have gone.
Ok. So my earlier question (in Slack, I think) about how "unstable" is unstable in Pharo is now answered. It's really unstable, not "Debian-like unstable".

It would also be nice to add some public info about that. I'll create an issue.

At the moment, I am stuck with GTDebugger, because it seems the old debugger does not have any keyboard shortcut form stepping through code. And that's something I would really really miss. So I'll suffer the pain of all the other glitches, and see how that goes.
Dimitris Chloupis
2016-01-16 10:12:29 UTC
Permalink
It really depend, when I was using pharo 4 back when it was the
"unstable/alpha" version it was rock solid, same story with pharo 3. Pharo
5 is the first seriously problematic unstable pharo version.

The reason is that Pharo 5 has a ton of core changes , new vm, new
debugger, new guis api , new compiler in short a ton of new things went in.
Even I that I love to working on the cutting edge of Pharo , I am forced to
go back to pharo 4. Shortcuts dont work, latest update broke the rendering
of menus, pharo 5 is a mess BUT when it becomes stable it will be our
biggest update by far dwarfing all previous updates combined.

Its even scarier if you take a look at the new stuff Pharo 6 will bring, I
dont envy Esteban :D
Post by David Allouche
Post by Ben Coman
So if occasionally if everything seems against you, after trying a
fresh Image as you already did, slide back to the last build that
worked for you (I don't think the issues you experience are Spur
related, but 50496 might be a good fallback as Pre-Spur and
Pre-GTDebugger). Then create a fresh Image from the latest build to
test your Slice and in that time maybe the other issues have gone.
Ok. So my earlier question (in Slack, I think) about how "unstable" is
unstable in Pharo is now answered. It's really unstable, not "Debian-like
unstable".
It would also be nice to add some public info about that. I'll create an issue.
At the moment, I am stuck with GTDebugger, because it seems the old
debugger does not have any keyboard shortcut form stepping through code.
And that's something I would really really miss. So I'll suffer the pain of
all the other glitches, and see how that goes.
stepharo
2016-01-16 11:12:14 UTC
Permalink
Hi david
Post by David Allouche
Post by Ben Coman
So if occasionally if everything seems against you, after trying a
fresh Image as you already did, slide back to the last build that
worked for you (I don't think the issues you experience are Spur
related, but 50496 might be a good fallback as Pre-Spur and
Pre-GTDebugger). Then create a fresh Image from the latest build to
test your Slice and in that time maybe the other issues have gone.
Ok. So my earlier question (in Slack, I think) about how "unstable" is unstable in Pharo is now answered. It's really unstable, not "Debian-like unstable".
No this is not really the case. In general we try hard to get alpha
stable after you get some hiccups.
This is not every day that we change FFI, GC, object internal
representations and the text editors, syntax highli
And our problems is that sometimes often we just jump over glitches
(this is why I'm getting mad because when I do videos
or other I see them in my face). Now I would love to have task forces
that check glitches.
Post by David Allouche
It would also be nice to add some public info about that. I'll create an issue.
At the moment, I am stuck with GTDebugger, because it seems the old debugger does not have any keyboard shortcut form stepping through code. And that's something I would really really miss. So I'll suffer the pain of all the other glitches, and see how that goes.
David Allouche
2016-01-16 11:22:33 UTC
Permalink
Post by David Allouche
Post by Ben Coman
So if occasionally if everything seems against you, after trying a
fresh Image as you already did, slide back to the last build that
worked for you (I don't think the issues you experience are Spur
related, but 50496 might be a good fallback as Pre-Spur and
Pre-GTDebugger). Then create a fresh Image from the latest build to
test your Slice and in that time maybe the other issues have gone.
Ok. So my earlier question (in Slack, I think) about how "unstable" is unstable in Pharo is now answered. It's really unstable, not "Debian-like unstable".
No this is not really the case. In general we try hard to get alpha stable after you get some hiccups.
This is not every day that we change FFI, GC, object internal representations and the text editors, syntax highli
And our problems is that sometimes often we just jump over glitches (this is why I'm getting mad because when I do videos
or other I see them in my face). Now I would love to have task forces that check glitches.
I get the picture now. I just checked in when the lobby, the roof and the plumbing of the hotel were undergoing renovations: it's not how it is usually.

I could question whether it's a good idea to cram so much in a single release cycle. But release management is a difficult topic, and often there is no good solution, just one that has less downsides than the others.

Cheers.
stepharo
2016-01-17 10:56:25 UTC
Permalink
Post by David Allouche
No this is not really the case. In general we try hard to get alpha stable after you get some hiccups.
This is not every day that we change FFI, GC, object internal representations and the text editors, syntax highli
And our problems is that sometimes often we just jump over glitches (this is why I'm getting mad because when I do videos
or other I see them in my face). Now I would love to have task forces that check glitches.
I get the picture now. I just checked in when the lobby, the roof and the plumbing of the hotel were undergoing renovations: it's not how it is usually.
I could question whether it's a good idea to cram so much in a single release cycle. But release management is a difficult topic, and often there is no good solution, just one that has less downsides than the others.
We do not control all the agendas. We were hesitating to get Pharo 50
without spur and for some people
it would have been sad. Then it would have meant 2 months in beta and
slow mode.
And Esteban worked a lot on the new FFI because it was needed: you do
not change an object header internal representation
without impact and since the previous FFI was generating assembly on the
fly and few people knew its implementation
and Igor got a depression we got stuck.
Then this is important to get the new FFI out because Ronie is coming to
visit us and he should work with Clement and esteban
on a faster version.
This is as simple as that. It means that all the libraries using FFI
would not have been available with Spur = no use for certain people like
roassal, moose, synectique.

Stef
Nicolai Hess
2016-01-15 22:19:51 UTC
Permalink
Post by David Allouche
Hello,
I believe the Contributing to Pharo page
<http://pharo.org/contribute-propose-fix>
http://pharo.org/contribute-propose-fix
is out of date.
Certainly. Time passes so fast.
Hm, I do this on a daily basis and it works for me.
Post by David Allouche
I found a bug
https://pharo.fogbugz.com/f/cases/17396/MonticelloRepositoryBrowser-example-MNU
and a made a fix in in a clean image (50525), created by the newly fixed PharoLauncher.
Then I followed the instruction on the page.
- Open the Monticello Browser.
- Open the Slice Maker by clicking the "Slice" button.
- In Slice Maker enter the Fogbugz issue number and "grab" the
issue summary.
- Select all dirty packages affected by your bug fix.
I have no idea how the "list of dirty packages" is generated, but
1. it shows a bunch of packages I have not touched
Sometimes MC is confused and some packages get dirty while they are not.
we had fixed a couple of bugs on how the repository version is searched and
loaded.
Sometimes it still loads (or compares against) the wrong version.
Deleting the package-cache (package-cache folder) may help
Post by David Allouche
1. it does not show the package that contains my fix (Spec-Tools)
This is strange. And MC shows the package as dirty? Then this is a bug.
I looked in the world menu, and I found this menu item called "Komitter".
That sounded like a good candidate.
Now, I guess I should select Pharo50Inbox as the Remote, write a commit
message, click Ok and hope for the best.
Sounds correct.
I'm not sure that I understand Komitter :).
Is that right?
I don't use komitter, I find it easier with the step described on the
how-to-contribute page
Post by David Allouche
I guess contributors are expected to use a recent image.
Yes it is always better and also to only publish one change at a time
(which is annoying when you have sequences of changes but
nice to track down changes).
So I think it would be good that someone updates the documentation. Or fix
the slice maker. Or both.
Can you reproduce the bug?
BTW it would be nice that you print and sign the pharo license, then you
send us a scanned or surface mail version
and we will archive it.
We ask that to every contributors.
Stef
Ben Coman
2016-01-16 01:15:51 UTC
Permalink
Post by David Allouche
Hello,
I believe the Contributing to Pharo page
http://pharo.org/contribute-propose-fix
is out of date.
I found a bug
https://pharo.fogbugz.com/f/cases/17396/MonticelloRepositoryBrowser-example-MNU
and a made a fix in in a clean image (50525), created by the newly fixed PharoLauncher.
Then I followed the instruction on the page.
- Open the Monticello Browser.
- Open the Slice Maker by clicking the "Slice" button.
- In Slice Maker enter the Fogbugz issue number and "grab" the
issue summary.
- Select all dirty packages affected by your bug fix.
I have no idea how the "list of dirty packages" is generated, but
1. it shows a bunch of packages I have not touched
2. it does not show the package that contains my fix (Spec-Tools)
I am curious on item 2. Was this just because the dirty package was sorted
to the top of the list (per Nicolai's snapshots)?
Post by David Allouche
I looked in the world menu, and I found this menu item called "Komitter".
That sounded like a good candidate.
I see how the term "Commit" is misleading here without context. Probably
"Press the <Save> button to commit your slice to the inbox repository"
would probably be better.
Post by David Allouche
Now, I guess I should select Pharo50Inbox as the Remote, write a commit
message, click Ok and hope for the best.
Is that right?
I guess contributors are expected to use a recent image. So I think it
would be good that someone updates the documentation. Or fix the slice
maker. Or both.
https://pharo.fogbugz.com/default.asp?17398

cheers -ben
David Allouche
2016-01-16 08:59:13 UTC
Permalink
Post by David Allouche
it shows a bunch of packages I have not touched
it does not show the package that contains my fix (Spec-Tools)
I am curious on item 2. Was this just because the dirty package was sorted to the top of the list (per Nicolai's snapshots)?
I think that was indeed the source of confusion.

But, really, this "list of dirty packages" should really only list dirty packages. What is the use case for selecting a non-dirty package?
Post by David Allouche
I looked in the world menu, and I found this menu item called "Komitter". That sounded like a good candidate.
I see how the term "Commit" is misleading here without context. Probably "Press the <Save> button to commit your slice to the inbox repository" would probably be better.
Yes, that would help, too.
Post by David Allouche
https://pharo.fogbugz.com/default.asp?17398 <https://pharo.fogbugz.com/default.asp?17398>
Cool!
stepharo
2016-01-16 11:14:18 UTC
Permalink
Post by David Allouche
Post by David Allouche
1. it shows a bunch of packages I have not touched
2. it does not show the package that contains my fix (Spec-Tools)
I am curious on item 2. Was this just because the dirty package was
sorted to the top of the list (per Nicolai's snapshots)?
I think that was indeed the source of confusion.
But, really, this "list of dirty packages" should really only list
dirty packages. What is the use case for selecting a non-dirty package?
because you never know
Sometimes you save a package then you do a slice and you should be able
to decide.
Post by David Allouche
Post by David Allouche
I looked in the world menu, and I found this menu item called
"Komitter". That sounded like a good candidate.
I see how the term "Commit" is misleading here without context.
Probably "Press the <Save> button to commit your slice to the inbox
repository" would probably be better.
Yes, that would help, too.
Post by David Allouche
https://pharo.fogbugz.com/default.asp?17398
Cool!
David Allouche
2016-01-16 11:25:07 UTC
Permalink
Post by stepharo
Post by David Allouche
But, really, this "list of dirty packages" should really only list dirty packages. What is the use case for selecting a non-dirty package?
because you never know
Sometimes you save a package then you do a slice and you should be able to decide.
That does not make sense to me. I guess it will, when I understand Monticello better.
Ben Coman
2016-01-16 13:44:34 UTC
Permalink
Post by David Allouche
Post by stepharo
Post by David Allouche
But, really, this "list of dirty packages" should really only list dirty packages. What is the use case for selecting a non-dirty package?
because you never know
Sometimes you save a package then you do a slice and you should be able to decide.
That does not make sense to me. I guess it will, when I understand Monticello better.
Its a very corner case. I've had to do it on occasion but I can't
exactly remember the circumstance. Now what might be useful is a
checkbox for displaying non-dirty packages - so we only display the
short list in the usual case. (??)

cheers -ben
Sven Van Caekenberghe
2016-01-16 14:46:47 UTC
Permalink
Post by Ben Coman
Post by David Allouche
Post by stepharo
Post by David Allouche
But, really, this "list of dirty packages" should really only list dirty packages. What is the use case for selecting a non-dirty package?
because you never know
Sometimes you save a package then you do a slice and you should be able to decide.
That does not make sense to me. I guess it will, when I understand Monticello better.
Its a very corner case. I've had to do it on occasion but I can't
exactly remember the circumstance. Now what might be useful is a
checkbox for displaying non-dirty packages - so we only display the
short list in the usual case. (??)
+100 it is very confusing right now
Post by Ben Coman
cheers -ben
Sean P. DeNigris
2016-01-22 17:50:16 UTC
Permalink
Post by Ben Coman
checkbox for displaying non-dirty packages - so we only display the
short list in the usual case. (??)
Ah! That would be very helpful. Particularly annoying is when you commit one
of your project's dirty packages, the list auto-scrolls to the package's new
location among the non-dirty packages, and you have to manually scroll back
to the top of the package list to find your next dirty package.



-----
Cheers,
Sean
--
View this message in context: http://forum.world.st/Contributing-to-Pharo-tp4871604p4873422.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
Dale Henrichs
2016-01-27 03:11:50 UTC
Permalink
I've been wondering no-one in the Pharo community has started building a
"Project Browser" to replace the "Package Browser" .... I've had a
"project browser" in tODE for several years now and you work with a
manageble list of Metacello projects like the following:

project list

instead of a giant list of packages like the following (only half of the
packages loaded in the image):

package list

Like a package browser, the project shows up as dirty whenever a package
in the project is dirtied and all of the operations `save`, `diff`, etc.
apply equally well to projects instead of packages .... with the added
bonus of performing the action on the the project itself instead of
having to view diffs on each dirty package or save each dirty package
... for the git-based repos, a single git commit is performed after all
of the dirty packages are committed , thus allowing git to track the
moves of methods between packages and combining all of the changes for a
unit of work in a single commit ...

Also, in the `project list`, the loaded version and blessing (1.0.5.4
[release]) or SHA of the git commit and branch name (b674ad0 [master])
is displayed:

Gofer 1.0.5.4 [release]
Grease b674ad0 [master]

... and this is very useful information. Metacello provides an api for
this information and a project-based workflow is just much simpler to
work in than a package-based workflow...

I'm willing to share code from tODE and Metacello if anyone is
interested ... and I think that a tool like this is really required to
properly support a git (or any other disk-based SCM) workflow ...

Perhaps work is already underway, but whenever I see folks talking about
the funky Monticello Browser interface, I can't help thinking about the
fact that I haven't used an interface like that for several years now
and it's time for the tooling to move forward a bit more:)

Dale
Post by Sean P. DeNigris
Post by Ben Coman
checkbox for displaying non-dirty packages - so we only display the
short list in the usual case. (??)
Ah! That would be very helpful. Particularly annoying is when you commit one
of your project's dirty packages, the list auto-scrolls to the package's new
location among the non-dirty packages, and you have to manually scroll back
to the top of the package list to find your next dirty package.
-----
Cheers,
Sean
--
View this message in context: http://forum.world.st/Contributing-to-Pharo-tp4871604p4873422.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
Christophe Demarey
2016-01-27 07:29:57 UTC
Permalink
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.

Cheers,
Christophe
Dale Henrichs
2016-01-27 14:13:56 UTC
Permalink
Haha, I certainly understand the lack of time/manpower ... Just in case
it wasn't clear, when I said "project browser" I wasn't talking about
replacing the code browser (Nautilus), but replacing/augmenting the
Monticello Browser - which should turn out to be a much simpler
undertaking --- "Project Browser" that manages the projects within the
image - load, save, changes, etc.:

project list screenshot


Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Sven Van Caekenberghe
2016-01-27 14:24:50 UTC
Permalink
Dale,

That looks like something very interesting and worth considering.

How easy is it to load tODE in a current (4.x 5.x) Pharo image ? I assume there is a Metacello configuration ;-) Is it possible to experiment with it in Pharo, independent from Gemstone ? I did not see it in the Catalog Browser (our current point & click installer).

Making tODE easily loadable would, IMHO, be the first step to a broader adaption of your work (in one form or another). The next step would be a getting started tutorial.

Sven
<Screen Shot 2016-01-27 at 6.11.33 AM.png>
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Dale Henrichs
2016-01-27 15:30:48 UTC
Permalink
Post by Sven Van Caekenberghe
Dale,
That looks like something very interesting and worth considering.
Thanks for the interest Sven!
Post by Sven Van Caekenberghe
How easy is it to load tODE in a current (4.x 5.x) Pharo image ? I assume there is a Metacello configuration ;-) Is it possible to experiment with it in Pharo, independent from Gemstone ? I did not see it in the Catalog Browser (our current point & click installer).
As it stands right now, tODE is not loadable/functional in Pharo ... In
my earlier tODE work, I implemented it in both Pharo and GemStone, but
this current version of tODE started out life as a remote development
environment for GemStone with emphasis on the remote aspect --- so a
significant chunk of tODE is devoted to managing windows running in a
Pharo image while executing all of the functionality in GemStone ...

With that said, I have architected tODE to isolate the basic tool
functionality from the GUI/command line and the model has evolved to the
point where all tools (both GUI and CommandLine --- yes there are
bash-like command mappings for all GUI functionality) share the same
basic algorithms behind a smalltalk api --- so developers can easily tap
into the functionality behind every GUI window/tODE shell command.

If you want to experiment today, you'll have to do that in GemStone with
GsDevKit_home[1]

[1] https://github.com/GsDevKit/GsDevKit_home
Post by Sven Van Caekenberghe
Making tODE easily loadable would, IMHO, be the first step to a broader adaption of your work (in one form or another). The next step would be a getting started tutorial.
Haha, I am bandwidth limited myself and as tODE and GsDevKit_home are
still works in progress (quite functional, but still evolving:) I do not
have the cycles to worry about getting tODE to work in Pharo.... I've
got my hands full with tODE and GemStone ...

Sooo at the end of the day, I would be willing to work with folks in
porting the interesting bits of tODE to Pharo, but "interesting bits" is
in the eye of the beholder and in this case I am not the beholder ...

I think it's more important to share some of the structural (on disk)
and architectural (in-image) lessons that I've learned about working
with tODE, git and Metacello ...

For example a "Project Browser" is a critical tool that needs to exist
... I've started sharing some of the implementation details with Thierry
already ... I would recommend that the "project browser" be implemented
in a set of classes totally independent of the GUI --- let the menu
items and functionality expressed in the "Project Browser" drive a
smalltalk api that can be called from a variety endpoints beyond the
initial "project browser" implementation (take the approach that
developers will want to incorporate all of the functionality in the
"project browser" in their own GT tool) so every menu item and every
meaningful click should call this "smalltalk api) and very little
"business logic" should be present in the GUI itself ...

The details of the functionality of the tool will depend upon the set of
work flows that will be supported and on this point I do have 4 years of
experience to bring to bear on this matter, but I want to share
information and opinions rather than dictate functionality: that is the
job of the tool builder:)

Since there seems to be interest, I intend to write a series of posts
describing the disk architecture and workflows used in GsDevKit_home for
incorporating git (or any other disk-based SCM) into smalltalk dev
environments ...

Dale
stepharo
2016-01-27 16:30:10 UTC
Permalink
We have the catalog. So how different it is.

And yes we would to change packages in nautilus.
Post by Dale Henrichs
Haha, I certainly understand the lack of time/manpower ... Just in
case it wasn't clear, when I said "project browser" I wasn't talking
about replacing the code browser (Nautilus), but replacing/augmenting
the Monticello Browser - which should turn out to be a much simpler
undertaking --- "Project Browser" that manages the projects within
project list screenshot
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Dale Henrichs
2016-01-27 17:23:44 UTC
Permalink
Post by stepharo
We have the catalog. So how different it is.
AFAIK the catalog only manages loads of projects ... to make it a
"Project Browser" you would add menu items for committing/saving
projects (i.e., save all dirty packages and for git repos, do a git
commit after updating packages); viewing the diffs for all packages in a
project; and of course the load functionality ....

As I said somewhere else, I don't think that it is a major job to
implement a "Project Browser", but it is something that really needs to
be done for supporting git-based projects. There is a natural 1-to-1
mapping between a Metacello project and it's git repository and as you
can see in the Git menu below, there are a number of additional
git-specific functions that need to be supported ...

Dale
Post by stepharo
And yes we would to change packages in nautilus.
Post by Dale Henrichs
Haha, I certainly understand the lack of time/manpower ... Just in
case it wasn't clear, when I said "project browser" I wasn't talking
about replacing the code browser (Nautilus), but replacing/augmenting
the Monticello Browser - which should turn out to be a much simpler
undertaking --- "Project Browser" that manages the projects within
project list screenshot
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
stepharo
2016-01-27 19:47:36 UTC
Permalink
I think that this is important that we do not think only in terms of git.
People will jump on the next cool and better because nicer versioning
system.
I think about fossil for example. I'm waiting for that. Because each
time I pair program with smart git users
I see how this is "simple". So I would really be against having git menu
in Pharo.
We should have an abstraction over the generic commands that a
decentralized versioning system should
offer.

Stef
Post by Dale Henrichs
Post by stepharo
We have the catalog. So how different it is.
AFAIK the catalog only manages loads of projects ... to make it a
"Project Browser" you would add menu items for committing/saving
projects (i.e., save all dirty packages and for git repos, do a git
commit after updating packages); viewing the diffs for all packages in
a project; and of course the load functionality ....
As I said somewhere else, I don't think that it is a major job to
implement a "Project Browser", but it is something that really needs
to be done for supporting git-based projects. There is a natural
1-to-1 mapping between a Metacello project and it's git repository and
as you can see in the Git menu below, there are a number of additional
git-specific functions that need to be supported ...
Dale
Post by stepharo
And yes we would to change packages in nautilus.
Post by Dale Henrichs
Haha, I certainly understand the lack of time/manpower ... Just in
case it wasn't clear, when I said "project browser" I wasn't talking
about replacing the code browser (Nautilus), but
replacing/augmenting the Monticello Browser - which should turn out
to be a much simpler undertaking --- "Project Browser" that manages
project list screenshot
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Dale Henrichs
2016-01-27 20:30:14 UTC
Permalink
Totally agree with having an abstraction layer or common set of
functions ...

As I've said before, I talk about git mainly because of github and
because there needs to be a "first" and I chose git ...

Other than details, I think that most if not all of the information I
plan to share is "disk-based SCM neutral" and applies equally to each
SCM ....

Dale
Post by stepharo
I think that this is important that we do not think only in terms of git.
People will jump on the next cool and better because nicer versioning
system.
I think about fossil for example. I'm waiting for that. Because each
time I pair program with smart git users
I see how this is "simple". So I would really be against having git
menu in Pharo.
We should have an abstraction over the generic commands that a
decentralized versioning system should
offer.
Stef
Post by Dale Henrichs
Post by stepharo
We have the catalog. So how different it is.
AFAIK the catalog only manages loads of projects ... to make it a
"Project Browser" you would add menu items for committing/saving
projects (i.e., save all dirty packages and for git repos, do a git
commit after updating packages); viewing the diffs for all packages
in a project; and of course the load functionality ....
As I said somewhere else, I don't think that it is a major job to
implement a "Project Browser", but it is something that really needs
to be done for supporting git-based projects. There is a natural
1-to-1 mapping between a Metacello project and it's git repository
and as you can see in the Git menu below, there are a number of
additional git-specific functions that need to be supported ...
Dale
Post by stepharo
And yes we would to change packages in nautilus.
Post by Dale Henrichs
Haha, I certainly understand the lack of time/manpower ... Just in
case it wasn't clear, when I said "project browser" I wasn't
talking about replacing the code browser (Nautilus), but
replacing/augmenting the Monticello Browser - which should turn out
to be a much simpler undertaking --- "Project Browser" that
project list screenshot
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
stepharo
2016-01-27 20:47:55 UTC
Permalink
So we are in sync then!
Post by Dale Henrichs
Totally agree with having an abstraction layer or common set of
functions ...
As I've said before, I talk about git mainly because of github and
because there needs to be a "first" and I chose git ...
Other than details, I think that most if not all of the information I
plan to share is "disk-based SCM neutral" and applies equally to each
SCM ....
Dale
Post by stepharo
I think that this is important that we do not think only in terms of git.
People will jump on the next cool and better because nicer versioning
system.
I think about fossil for example. I'm waiting for that. Because each
time I pair program with smart git users
I see how this is "simple". So I would really be against having git
menu in Pharo.
We should have an abstraction over the generic commands that a
decentralized versioning system should
offer.
Stef
Post by Dale Henrichs
Post by stepharo
We have the catalog. So how different it is.
AFAIK the catalog only manages loads of projects ... to make it a
"Project Browser" you would add menu items for committing/saving
projects (i.e., save all dirty packages and for git repos, do a git
commit after updating packages); viewing the diffs for all packages
in a project; and of course the load functionality ....
As I said somewhere else, I don't think that it is a major job to
implement a "Project Browser", but it is something that really needs
to be done for supporting git-based projects. There is a natural
1-to-1 mapping between a Metacello project and it's git repository
and as you can see in the Git menu below, there are a number of
additional git-specific functions that need to be supported ...
Dale
Post by stepharo
And yes we would to change packages in nautilus.
Post by Dale Henrichs
Haha, I certainly understand the lack of time/manpower ... Just in
case it wasn't clear, when I said "project browser" I wasn't
talking about replacing the code browser (Nautilus), but
replacing/augmenting the Monticello Browser - which should turn
out to be a much simpler undertaking --- "Project Browser" that
project list screenshot
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Esteban A. Maringolo
2016-01-27 20:47:26 UTC
Permalink
Totally agree with having an abstraction layer or common set of functions ...
As I've said before, I talk about git mainly because of github and because there needs to be a "first" and I chose git ...
Other than details, I think that most if not all of the information I plan to share is "disk-based SCM neutral" and applies equally to each SCM ....
AFAIU FileTree has that purpose, although I don't know if it is used
with something other than Git as a SCM.


Esteban A. Maringolo
Dale Henrichs
2016-01-27 21:08:53 UTC
Permalink
Esteban,

FileTree is indeed neutral with regards to SCM, but Stef and I were
referring to a smalltalk api for managing the git (or other repos) in
terms of doing commits, checkouts and the like ... FileTree itself isn't
involved ...

Dale
Post by Esteban A. Maringolo
Totally agree with having an abstraction layer or common set of functions ...
As I've said before, I talk about git mainly because of github and because there needs to be a "first" and I chose git ...
Other than details, I think that most if not all of the information I plan to share is "disk-based SCM neutral" and applies equally to each SCM ....
AFAIU FileTree has that purpose, although I don't know if it is used
with something other than Git as a SCM.
Esteban A. Maringolo
Christophe Demarey
2016-01-28 08:23:33 UTC
Permalink
I read too fast.
But my comment is still true.
We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.
Esteban did a prototype for himself but he has no time to work on it, sadly
<Screen Shot 2016-01-27 at 6.11.33 AM.png>
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Sven Van Caekenberghe
2016-01-28 08:34:30 UTC
Permalink
Post by Christophe Demarey
I read too fast.
But my comment is still true.
We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.
It is totally wrong to put Git and Monticello at the same level, they are related but different things.

Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage.

Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes.

But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.
Post by Christophe Demarey
Esteban did a prototype for himself but he has no time to work on it, sadly
<Screen Shot 2016-01-27 at 6.11.33 AM.png>
Dale
Post by Christophe Demarey
Hi Dale,
Of course, we have that in mind. We would love to have it but lack of time / manpower ...
We are not happy with the current browser (Nautilus). We could have something really more powerful. It will come but we have other projects to achieve before ;)
If someone comes with a better browser, we would love to set it as default.
Cheers,
Christophe
Christophe Demarey
2016-01-28 08:55:37 UTC
Permalink
Post by Sven Van Caekenberghe
Post by Christophe Demarey
I read too fast.
But my comment is still true.
We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.
It is totally wrong to put Git and Monticello at the same level, they are related but different things.
Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage.
Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes.
But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.
I fully agree.
The problem is that Monticello does too much things: code representation, versioning, loading of the code and call of #initialize methods, etc.
We want to keep a code representation but the question that comes: is it the better code representation? We also have ring, ficus (hierarchical model on top of ring). For code representation, we would like to represent code (from packages to method source code) for multiple versions, in a hierarchical way (to be able to query easily), potentially coming from a remote environment.
We also need to split the loading part (loading from the code representation) and the serialization / deserialization part.

At the end, we want an uniform way to represent code (not SCM related), a common API for versioning and different backends (git, Monticello). By the way, we can add the storage format: there are already different versions of the filetree format (with metadata, without, one file per method, one file per class). At some point, we need to handle this point of variability.
Sven Van Caekenberghe
2016-01-28 08:57:48 UTC
Permalink
Post by Christophe Demarey
Post by Sven Van Caekenberghe
Post by Christophe Demarey
I read too fast.
But my comment is still true.
We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.
It is totally wrong to put Git and Monticello at the same level, they are related but different things.
Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage.
Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes.
But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.
I fully agree.
The problem is that Monticello does too much things: code representation, versioning, loading of the code and call of #initialize methods, etc.
We want to keep a code representation but the question that comes: is it the better code representation? We also have ring, ficus (hierarchical model on top of ring). For code representation, we would like to represent code (from packages to method source code) for multiple versions, in a hierarchical way (to be able to query easily), potentially coming from a remote environment.
We also need to split the loading part (loading from the code representation) and the serialization / deserialization part.
At the end, we want an uniform way to represent code (not SCM related), a common API for versioning and different backends (git, Monticello). By the way, we can add the storage format: there are already different versions of the filetree format (with metadata, without, one file per method, one file per class). At some point, we need to handle this point of variability.
Excellent summary. Lots of work ;-)
Dale Henrichs
2016-01-28 20:28:03 UTC
Permalink
Post by Christophe Demarey
Post by Sven Van Caekenberghe
Post by Christophe Demarey
I read too fast.
But my comment is still true.
We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.
It is totally wrong to put Git and Monticello at the same level, they are related but different things.
Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage.
Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes.
But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.
I fully agree.
The problem is that Monticello does too much things: code representation, versioning, loading of the code and call of #initialize methods, etc.
We want to keep a code representation but the question that comes: is it the better code representation? We also have ring, ficus (hierarchical model on top of ring). For code representation, we would like to represent code (from packages to method source code) for multiple versions, in a hierarchical way (to be able to query easily), potentially coming from a remote environment.
We also need to split the loading part (loading from the code representation) and the serialization / deserialization part.
At the end, we want an uniform way to represent code (not SCM related), a common API for versioning and different backends (git, Monticello).
Agreed ....
Post by Christophe Demarey
By the way, we can add the storage format: there are already different versions of the filetree format (with metadata, without, one file per method, one file per class). At some point, we need to handle this point of variability.
On this point I think that we will always want to support multiple disk
formats .... in a perfect world, there is no reason not to have one file
per method and no Monticello meta data ... but perfection does not exist.

Monticello meta data was included in FileTree because it was needed over
the last 4 years so that developers could dip there toes into using git
without completely abandoning Monticello .... Now that we are gaining a
critical mass of users and the tool support is starting to show up,
dropping the meta data is a practical thing to do ... but there is code
committed on github that hasn't been updated in years that is still
being used and Monticello meta data is still present, so it is necessary
to "support the older formats".

The one file per method was chosen for two reasons.

The first reason has to do with the fact that disk based SCMs are
designed manage files as atomic units and in Smalltalk the method is the
atomic unit for source code ... it is very convenient to be able to
leverage native SCN files for a) versioning files and b) maintaining
version history for a method as it moves from class to class ...

The second reason has to do with the fact that there is no cross
platform fileout format for Smalltalk. Chunk format comes is "common"
but chunk format also relies on executing Smalltalk code to define
classes and there hasn't been a standard class creation method since the
second Smalltalk implementation showed up on the planet ... Technically,
class and method definitions can be derived from a file per class
approach, but without a standardized file format you end up with each
dialect isolating itself from the others and this is not desirable nor
necessary ... It just takes more work to come up with a file format that
can accommodate the different needs of the different dialects .... the
one method per file and a generic properties file for class
definitions[1] and the provision for allowing additional platform
specific property files and directories (like monticello meta data)
simplified the whole process ...

So I would like changing the disk format for sharing source code to be a
multi-platform effort - which means that format that is chosen is
flexible enough to allow dialects to include the meta data that they
need ... It is important to note that there has never been a guarantees
that all platforms would PRESERVE and WRITE foreign meta data ... the
only provision that exists is that ALL platforms can READ the format
while ignoring anything it doesn't recognize ....

Dale

[1]
Loading Image...
Bernhard Pieber
2016-01-29 17:02:30 UTC
Permalink
Hi Dale,

I am trying to understand this a little better. If a package containing metadata would be changed using a dialect which cannot interpret the metadata, wouldn’t or at least couldn’t it be broken or lost afterwards for a dialect which tries to interpret the metadata? At least if my assumption is correct that the metadata is related to the code in some way?

Bernhard
Post by Christophe Demarey
By the way, we can add the storage format: there are already different versions of the filetree format (with metadata, without, one file per method, one file per class). At some point, we need to handle this point of variability.
On this point I think that we will always want to support multiple disk formats .... in a perfect world, there is no reason not to have one file per method and no Monticello meta data ... but perfection does not exist.
Monticello meta data was included in FileTree because it was needed over the last 4 years so that developers could dip there toes into using git without completely abandoning Monticello .... Now that we are gaining a critical mass of users and the tool support is starting to show up, dropping the meta data is a practical thing to do ... but there is code committed on github that hasn't been updated in years that is still being used and Monticello meta data is still present, so it is necessary to "support the older formats".
The one file per method was chosen for two reasons.
The first reason has to do with the fact that disk based SCMs are designed manage files as atomic units and in Smalltalk the method is the atomic unit for source code ... it is very convenient to be able to leverage native SCN files for a) versioning files and b) maintaining version history for a method as it moves from class to class ...
The second reason has to do with the fact that there is no cross platform fileout format for Smalltalk. Chunk format comes is "common" but chunk format also relies on executing Smalltalk code to define classes and there hasn't been a standard class creation method since the second Smalltalk implementation showed up on the planet ... Technically, class and method definitions can be derived from a file per class approach, but without a standardized file format you end up with each dialect isolating itself from the others and this is not desirable nor necessary ... It just takes more work to come up with a file format that can accommodate the different needs of the different dialects .... the one method per file and a generic properties file for class definitions[1] and the provision for allowing additional platform specific property files and directories (like monticello meta data) simplified the whole process ...
So I would like changing the disk format for sharing source code to be a multi-platform effort - which means that format that is chosen is flexible enough to allow dialects to include the meta data that they need ... It is important to note that there has never been a guarantees that all platforms would PRESERVE and WRITE foreign meta data ... the only provision that exists is that ALL platforms can READ the format while ignoring anything it doesn't recognize ....
Dale
[1] https://raw.githubusercontent.com/CampSmalltalk/Cypress/master/img/CypressStructure-STIC2012.png
Dale Henrichs
2016-01-29 18:12:52 UTC
Permalink
Post by Christophe Demarey
Hi Dale,
I am trying to understand this a little better. If a package containing metadata would be changed using a dialect which cannot interpret the metadata, wouldn’t or at least couldn’t it be broken or lost afterwards for a dialect which tries to interpret the metadata? At least if my assumption is correct that the metadata is related to the code in some way?
Bernhard,

Very good question.

You are absolutely correct, the act of writing a new version of a
package using a different dialect would indeed destroy the
platform-specific meta data.

However, it is NOT expected that two different dialects (with different
disk formats) share the same commit.

If dialect A has committed a package, then dialect B is expected to be
able to read the package written by dialect A. Being able to read the
package means that all of the information that is common between the two
dialects will be preserved ... in FileTree, this means that:

- all of the class and instance method source is readable and shared
- the method category is readable and shared
- the name, superclass, instance variables, class instance variables,
pools, class category, and class vars for classes is readable and
shared

This is an awful lot common data .... things like traits or namespaces
are dialect specific and are ignored by dialects that don't have them
when reading the package ...

When dialect B has read, loaded/compiled and tested the code in the
package, the developer has a couple of choices to be made when writing
out a new package with her changes:

- use the current branch
- use a new branch

I think that using a new branch is the cleanest option. When I port a
project like Zinc[1] to GemStone[2], the master branch is preserved for
the Pharo-specific code. I create a gs_master branch where the GemStone
code goes. When Sven commits new code in his repo, I merge the changes
from the his new master branch into my gs_master and resolve conflicts
if any, run tests and I'm done.

The vast mjority of the code base is shared between GemStone and Pharo
and only the places where I made changes in porting Zinc to GemStone
have the potential for conflicts. Running the tests should highlight
any impacts not covered by direct conflicts and if there were some
Pharo-specific meta data that I deleted on my branch that causes a test
to fail then that is up to me to research and fix.

When I have changes to feed back to Sven, I make those changes on a
separate "topic branch"[3] that is merged into gs_master for my use. I
then cherry-pick[4] the topic branch commit into a separate topic branch
off of the master branch (this picks up only the changes I made on the
topic branch) and open a pull request[5] against Svens repository ... if
Sven were using travis-ci, tests would be run automatically against a
merge of my changes into his master branch ... if the tests are green
and the code passes Sven's scrutiny in a code review he merges my
proposed changes into his master branch...

GemStone and Pharo share a common disk format (not 100% common), but
since git only merges the deltas for a commit, it is relatively easy to
keep the meta data differences isolated while still sharing the vast
bulk of the code and classes ....

Dale

[1] https://github.com/svenvc/zinc
[2] https://github.com/GsDevKit/zinc
[3] https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows
[4]
http://think-like-a-git.net/sections/rebase-from-the-ground-up/cherry-picking-explained.html
[5] https://help.github.com/articles/using-pull-requests/
David Allouche
2016-01-29 22:45:53 UTC
Permalink
Thanks Dale for all the explanations.

How Monticello and version control relate in the big picture is starting to make sense for me.

Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.

I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.

Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control.

Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model.

I have a lot of ideas to improve browsing and source code management in Pharo. I can make no promises, but I would like to produce something there.
Eliot Miranda
2016-01-30 01:17:54 UTC
Permalink
Hi David,
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is starting to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.
I agree so strongly. Class file outs which are eg sorted by selector make much more sense. They won't hit the file name length limit. They make it trivial to maintain method and class comment time stamps. They're easier to construct into snapshots because it's easier to decode the file name.

And then it's easy to add files for package load/unload scripts and for the history. And then one is much more decoupled from the specific back end. It could be mercurial just as easily as git.
Post by David Allouche
I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control.
This is a good analysis. What's valuable to the Pharo community is not displacing an already functional dvcs (Monticello) with an ill-suited one (git), but in being able to function in ecosystems like github where people can display their identity and where infrastructure for bug reports etc exist.
Post by David Allouche
Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model.
Absolutely. Dale's talk of ditching Monticello metadata fills me with repulsion and makes me want to ask is he trying to sabotage or what? It seems entirely destructive. We have a functional package manager which currently supports interchange between Pharo, Squeak and Cuis, something that I think is very important and valuable. We should have the confidence to improve on it and work on exchanging improvements between the dialects. For example Bert Freudenburg recently extended the commit dialog with the ability to mark changes to ignore, which are then /not/ committed, which makes it possible to maintain a few of one's favorite image mods without the tedium of reverting them (or whatever other Sisyphean means) to commit from ones work image.
Post by David Allouche
I have a lot of ideas to improve browsing and source code management in Pharo. I can make no promises, but I would like to produce something there.
Good luck in your efforts.


_,,,^..^,,,_ (phone)
Best, Eliot
Dale Henrichs
2016-02-02 19:35:35 UTC
Permalink
Post by Eliot Miranda
Hi David,
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is starting to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.
I agree so strongly. Class file outs which are eg sorted by selector make much more sense. They won't hit the file name length limit. They make it trivial to maintain method and class comment time stamps. They're easier to construct into snapshots because it's easier to decode the file name.
And then it's easy to add files for package load/unload scripts and for the history. And then one is much more decoupled from the specific back end. It could be mercurial just as easily as git.
Post by David Allouche
I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control.
This is a good analysis. What's valuable to the Pharo community is not displacing an already functional dvcs (Monticello) with an ill-suited one (git), but in being able to function in ecosystems like github where people can display their identity and where infrastructure for bug reports etc exist.
Post by David Allouche
Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model.
Absolutely. Dale's talk of ditching Monticello metadata fills me with repulsion and makes me want to ask is he trying to sabotage or what?
Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how
about you assume that I'm doing "or what":)

The Monticello metadata in a git repository is redundant and leads to
unnecessary commit conflicts -- end of story ....

Despite the fact that the Monticello metadata is redundant, I have made
sure that the Monticello metadata was included in FileTree from the very
beginning for the very reason that I wanted developers to be able to try
out FileTree, git and github without having to burn any Monticello
bridges .... if they didn't like FileTree, git and github, then they
would be able to back out of their use of git without losing data ...
Post by Eliot Miranda
It seems entirely destructive.
It is not destructive ...
Post by Eliot Miranda
We have a functional package manager which currently supports interchange between Pharo, Squeak and Cuis,
and GemStone?

I assume that you are talking about Monticello packages and Monticello
repositories ... or what?

I am really not trying to do anything but "invent the future" --- I am
not trying to destroy, I am trying to improve ... If you are not able to
see the shortcomings of Monticello repositories (Note that I am
distinguishing between Monticello packages and Monticello repositories
--- FileTree uses Monticello packages and replaces Monticello
repositories with git) and where git has advantages over Monticello
repositories, then you should continue to use Monticello repositories ...

Personally I don't see Monticello repositories going away anytime soon
and expect to support Monticello repositories in GsDevKit_home, tODE,
and Metacello for the rest of my life:)

Dale
Eliot Miranda
2016-02-02 20:54:59 UTC
Permalink
Hi Dale,

On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs <
Post by Dale Henrichs
Post by Eliot Miranda
Hi David,
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is starting
to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method
format, even though that is relatively hostile to git user interfaces
optimised for other languages. There is really a need for a file-per-class
exchange format, because that would works a lot better with the existing
VCS ecosystem.
I agree so strongly. Class file outs which are eg sorted by selector
make much more sense. They won't hit the file name length limit. They
make it trivial to maintain method and class comment time stamps. They're
easier to construct into snapshots because it's easier to decode the file
name.
And then it's easy to add files for package load/unload scripts and for
the history. And then one is much more decoupled from the specific back
end. It could be mercurial just as easily as git.
I think more package-based user interfaces would indeed be a very good
Post by David Allouche
idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular because it is
a new shiny toy. I disagree with this idea. Git is a typical
worse-is-better tool. It's good enough for most people, but it still has
many shortcomings. It is popular in spite of its shortcomings. It became
popular as destination for projects shifting from CVS and Subversion. So it
is unlikely to be displaced by a newer, incrementally shinier tools.
Anything that will displace it will have to provide an improvement of a
similar magnitude as the jump between centralised and distributed version
control.
This is a good analysis. What's valuable to the Pharo community is not
displacing an already functional dvcs (Monticello) with an ill-suited one
(git), but in being able to function in ecosystems like github where people
can display their identity and where infrastructure for bug reports etc
exist.
Still, I think it's a good idea not to restrict high level models to what
Post by David Allouche
git provides if that's a less than ideal fit to the image model.
Absolutely. Dale's talk of ditching Monticello metadata fills me with
repulsion and makes me want to ask is he trying to sabotage or what?
Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how
about you assume that I'm doing "or what":)
The Monticello metadata in a git repository is redundant and leads to
unnecessary commit conflicts -- end of story ....
No it's /not/ the end of the story. The essential part of the story is how
Monticello remains compatible and interoperable between dialects. I
haven't seen you account for how you maintain that compatibility. As far
as I can tell, you propose replacing the Monticello metadata with that from
git. How do I, as a Squeak user with Monticello, ever get to look at your
package again? As I understand it, moving the metadata from Monticello
commit time to git means that the metadata is in a format that git
determines, not Monticello.

So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary commit
conflicts -- end of story ....", which implies you want to eliminate the
Monticello metadata, and on the other hand you say you're keeping the
Monticello metadata. I'm hopelessly confused. How does the Monticello
metadata get reconstituted if it's been thrown away?

What happens to the metadata in the following workflow?

load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello


Despite the fact that the Monticello metadata is redundant, I have made
Post by Dale Henrichs
sure that the Monticello metadata was included in FileTree from the very
beginning for the very reason that I wanted developers to be able to try
out FileTree, git and github without having to burn any Monticello bridges
.... if they didn't like FileTree, git and github, then they would be able
to back out of their use of git without losing data ...
Then forgive me. I couldn't see the wood for the trees. When I read your
talk of eliminating the conflicts from git commits due to the Monticello
metadata I infer that you're eliminating the Monticello metadata. I'm not
sure I understand the implications of this, but it seems to me that a
natural consequence is that the Montivcello metadata is lost and at that
point merges and the like become problematic. What am I missing?
Post by Dale Henrichs
It seems entirely destructive.
It is not destructive ...
Post by Eliot Miranda
We have a functional package manager which currently supports interchange
between Pharo, Squeak and Cuis,
and GemStone?
I assume that you are talking about Monticello packages and Monticello
repositories ... or what?
Yes.
Post by Dale Henrichs
I am really not trying to do anything but "invent the future" --- I am not
trying to destroy, I am trying to improve ... If you are not able to see
the shortcomings of Monticello repositories (Note that I am distinguishing
between Monticello packages and Monticello repositories --- FileTree uses
Monticello packages and replaces Monticello repositories with git) and
where git has advantages over Monticello repositories, then you should
continue to use Monticello repositories ...
But what does this imply to some package that starts off in a Monticello
repository and then spends some time in gitland? Can I merge again? If I
can I'm happy. If I can't, I feel sabotaged.
Post by Dale Henrichs
Personally I don't see Monticello repositories going away anytime soon and
expect to support Monticello repositories in GsDevKit_home, tODE, and
Metacello for the rest of my life:)
Fine. Except merging is, IIUC, about method time stamps and ancestry. If
that gets preserved then I'm happy. But for the life of me I haven't read
an explanation that reassures me that these are being preserved. Do you
see the roots of my fear?

Dale
_,,,^..^,,,_
confused, Eliot
Dale Henrichs
2016-02-02 23:00:44 UTC
Permalink
Post by Christophe Demarey
Hi Dale,
On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs
Hi David,
On Jan 29, 2016, at 2:45 PM, David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big
picture is starting to make sense for me.
Now, I better understand why filetree ended up uses a
file-per-method format, even though that is relatively
hostile to git user interfaces optimised for other
languages. There is really a need for a file-per-class
exchange format, because that would works a lot better
with the existing VCS ecosystem.
I agree so strongly. Class file outs which are eg sorted by
selector make much more sense. They won't hit the file name
length limit. They make it trivial to maintain method and
class comment time stamps. They're easier to construct into
snapshots because it's easier to decode the file name.
And then it's easy to add files for package load/unload
scripts and for the history. And then one is much more
decoupled from the specific back end. It could be mercurial
just as easily as git.
I think more package-based user interfaces would indeed be
a very good idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular
because it is a new shiny toy. I disagree with this idea.
Git is a typical worse-is-better tool. It's good enough
for most people, but it still has many shortcomings. It is
popular in spite of its shortcomings. It became popular as
destination for projects shifting from CVS and Subversion.
So it is unlikely to be displaced by a newer,
incrementally shinier tools. Anything that will displace
it will have to provide an improvement of a similar
magnitude as the jump between centralised and distributed
version control.
This is a good analysis. What's valuable to the Pharo
community is not displacing an already functional dvcs
(Monticello) with an ill-suited one (git), but in being able
to function in ecosystems like github where people can display
their identity and where infrastructure for bug reports etc exist.
Still, I think it's a good idea not to restrict high level
models to what git provides if that's a less than ideal
fit to the image model.
Absolutely. Dale's talk of ditching Monticello metadata fills
me with repulsion and makes me want to ask is he trying to
sabotage or what?
Eliot, you can't be serious - accusing me of sabotage? Ah,
well.... how about you assume that I'm doing "or what":)
The Monticello metadata in a git repository is redundant and leads
to unnecessary commit conflicts -- end of story ....
No it's /not/ the end of the story. The essential part of the story
is how Monticello remains compatible and interoperable between
dialects. I haven't seen you account for how you maintain that
compatibility. As far as I can tell, you propose replacing the
Monticello metadata with that from git. How do I, as a Squeak user
with Monticello, ever get to look at your package again? As I
understand it, moving the metadata from Monticello commit time to git
means that the metadata is in a format that git determines, not
Monticello.
Good question, FileTree has been supported on Squeak since the very
beginning (I along with a small number of Squeak users have made sure of
that).

So TODAY, any Squeak user can "look at, load and commit" any package
that has been written using FileTree (with or without Monticello meta
data).

[1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak
Post by Christophe Demarey
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say
you're keeping the Monticello metadata. I'm hopelessly confused. How
does the Monticello metadata get reconstituted if it's been thrown away?
Monticello meta data is not an integral part of the "package-ness" of a
Monticello package ... it _is_ integral to the "repository-ness" of a
Monticello package ...

If the Monticello metadata is "thrown away" then the revision history
for Monticello is lost, but for a package that is "born" in a git
repository, the Monticello metadata is not needed. Git has it's own
commit meta data and the Monticello metadata is redundant.

If you want to see the revision history from a git-based FileTree repo,
then:

1. one can include the Monticello meta data as part of the package -
this is what FileTree
currently does
2. one can build a tool that reconstructs the Monticello version
history from the git metadata
making it possible to use "old" Monitcello tools to look at the
git repo - I believe that Thierry's
GitFileTree takes this approach for metadata-less repositories
3. One can build a new tool that presents the git metadata without
reconstructing the
Monticello metadata at all. Note that by "embracing git", it is
possible to present revision
history at the package level (current mcz techology) as well as
the class and method level -
which is what I do in tODE
Post by Christophe Demarey
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
The above workflow can be accomplished whether or not Monticello
metadata is present, however, if one does not make an effort to preserve
the revision history then at the end of your workflow the Monticello
metadata is lost.

If one takes the pains to preserve the Monticello metadata before
committing to the git repository and the metadata is updated with each
commit during the git lifetime, then the full metadata will be present
at the end ...

This I think is the crux of the discussion.

There are a number of alternate schemes that can be used to preserve the
metadata through this scenario:

1. (current FileTree implementation) store the Monticello meta data
in git and update on
every git commit
2. duplicate the existing Monticello revision history by committing
in order all of the
package ancestors and arrange for a way to reconstruct Monticello
metadata from git
meta data
3. (variant of 1) store the original monticello meta data in a file,
do not update on every commit
but arrange for a way to reconstruct Monticello metadata from git
meta data and graft onto
original Monticello meta data for use in mcz repository --- on demand
4. ????

Option 3 seems to be a good compromise solution and perhaps is the
approach that should be adopted moving forward ... we get to preserver
Monticello metadata while avoiding the messy commit conflicts for git
while providing a (somewhat) seamless path for a package to migrate back
into the mcz repository world ... if we somehow incorporate the SHA of
the commit and the github/bitbucket url into the revision history, then
it would be possible to perform a 3 way merge involving two mcz package
versions and common ancestor that is only present in git ...
Post by Christophe Demarey
Despite the fact that the Monticello metadata is redundant, I have
made sure that the Monticello metadata was included in FileTree
from the very beginning for the very reason that I wanted
developers to be able to try out FileTree, git and github without
having to burn any Monticello bridges .... if they didn't like
FileTree, git and github, then they would be able to back out of
their use of git without losing data ...
Then forgive me. I couldn't see the wood for the trees. When I read
your talk of eliminating the conflicts from git commits due to the
Monticello metadata I infer that you're eliminating the Monticello
metadata. I'm not sure I understand the implications of this, but it
seems to me that a natural consequence is that the Montivcello
metadata is lost and at that point merges and the like become
problematic. What am I missing?
You've hit the nail on the head, but I think that option 3 above gives
us a way to avoid losing the Monticello metadata without incurring a hit
for a packages lifetime while in git ...

A package that starts its life in git will have an empty Monticello
metadata and a package that never makes it's way into an mcz repository
will not incur per commit penalties ...
Post by Christophe Demarey
It seems entirely destructive.
It is not destructive ...
We have a functional package manager which currently supports
interchange between Pharo, Squeak and Cuis,
and GemStone?
I assume that you are talking about Monticello packages and
Monticello repositories ... or what?
Yes.
I am really not trying to do anything but "invent the future" ---
I am not trying to destroy, I am trying to improve ... If you are
not able to see the shortcomings of Monticello repositories (Note
that I am distinguishing between Monticello packages and
Monticello repositories --- FileTree uses Monticello packages and
replaces Monticello repositories with git) and where git has
advantages over Monticello repositories, then you should continue
to use Monticello repositories ...
But what does this imply to some package that starts off in a
Monticello repository and then spends some time in gitland? Can I
merge again? If I can I'm happy. If I can't, I feel sabotaged.
and sabotage was never my intent ...
Post by Christophe Demarey
Personally I don't see Monticello repositories going away anytime
soon and expect to support Monticello repositories in
GsDevKit_home, tODE, and Metacello for the rest of my life:)
Fine. Except merging is, IIUC, about method time stamps and
ancestry. If that gets preserved then I'm happy. But for the life of
me I haven't read an explanation that reassures me that these are
being preserved. Do you see the roots of my fear?
Haha, from the very beginning back in 2012, I understood that there
would be fear and resistance to change and anger and joy and excitement
but it was not clear when if ever we'd reach point where a resolution
for this "problem" was needed: either no one would be interested or
everyone would be interested or ???

I really think that option 3 is going to be the best compromise moving
forward - there is some implementation work that will be required but I
really think option 3 gives you (and frankly me) a way to preserve the
Monticello revision history for packages that make their way back and
forth between lifetimes in git and Monticello repositories.

Eliot, I appreciate the fact that you demanded a better solution!

Dale
Ben Coman
2016-02-03 05:00:25 UTC
Permalink
On Wed, Feb 3, 2016 at 7:00 AM, Dale Henrichs
Post by Christophe Demarey
Hi Dale,
On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs
Post by Dale Henrichs
Post by Eliot Miranda
Hi David,
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is starting
to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method
format, even though that is relatively hostile to git user interfaces
optimised for other languages. There is really a need for a file-per-class
exchange format, because that would works a lot better with the existing VCS
ecosystem.
I agree so strongly. Class file outs which are eg sorted by selector
make much more sense. They won't hit the file name length limit. They make
it trivial to maintain method and class comment time stamps. They're easier
to construct into snapshots because it's easier to decode the file name.
And then it's easy to add files for package load/unload scripts and for
the history. And then one is much more decoupled from the specific back
end. It could be mercurial just as easily as git.
Post by David Allouche
I think more package-based user interfaces would indeed be a very good
idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular because it is
a new shiny toy. I disagree with this idea. Git is a typical worse-is-better
tool. It's good enough for most people, but it still has many shortcomings.
It is popular in spite of its shortcomings. It became popular as destination
for projects shifting from CVS and Subversion. So it is unlikely to be
displaced by a newer, incrementally shinier tools. Anything that will
displace it will have to provide an improvement of a similar magnitude as
the jump between centralised and distributed version control.
This is a good analysis. What's valuable to the Pharo community is not
displacing an already functional dvcs (Monticello) with an ill-suited one
(git), but in being able to function in ecosystems like github where people
can display their identity and where infrastructure for bug reports etc
exist.
Post by David Allouche
Still, I think it's a good idea not to restrict high level models to
what git provides if that's a less than ideal fit to the image model.
Absolutely. Dale's talk of ditching Monticello metadata fills me with
repulsion and makes me want to ask is he trying to sabotage or what?
Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how
about you assume that I'm doing "or what":)
The Monticello metadata in a git repository is redundant and leads to
unnecessary commit conflicts -- end of story ....
No it's /not/ the end of the story. The essential part of the story is how
Monticello remains compatible and interoperable between dialects. I haven't
seen you account for how you maintain that compatibility. As far as I can
tell, you propose replacing the Monticello metadata with that from git. How
do I, as a Squeak user with Monticello, ever get to look at your package
again? As I understand it, moving the metadata from Monticello commit time
to git means that the metadata is in a format that git determines, not
Monticello.
Good question, FileTree has been supported on Squeak since the very
beginning (I along with a small number of Squeak users have made sure of
that).
So TODAY, any Squeak user can "look at, load and commit" any package that
has been written using FileTree (with or without Monticello meta data).
[1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary commit
conflicts -- end of story ....", which implies you want to eliminate the
Monticello metadata, and on the other hand you say you're keeping the
Monticello metadata. I'm hopelessly confused. How does the Monticello
metadata get reconstituted if it's been thrown away?
Monticello meta data is not an integral part of the "package-ness" of a
Monticello package ... it _is_ integral to the "repository-ness" of a
Monticello package ...
If the Monticello metadata is "thrown away" then the revision history for
Monticello is lost, but for a package that is "born" in a git repository,
the Monticello metadata is not needed. Git has it's own commit meta data and
the Monticello metadata is redundant.
If you want to see the revision history from a git-based FileTree repo,
1. one can include the Monticello meta data as part of the package - this
is what FileTree
currently does
2. one can build a tool that reconstructs the Monticello version history
from the git metadata
making it possible to use "old" Monitcello tools to look at the git
repo - I believe that Thierry's
GitFileTree takes this approach for metadata-less repositories
3. One can build a new tool that presents the git metadata without
reconstructing the
Monticello metadata at all. Note that by "embracing git", it is
possible to present revision
history at the package level (current mcz techology) as well as the
class and method level -
which is what I do in tODE
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
The above workflow can be accomplished whether or not Monticello metadata is
present, however, if one does not make an effort to preserve the revision
history then at the end of your workflow the Monticello metadata is lost.
If one takes the pains to preserve the Monticello metadata before committing
to the git repository and the metadata is updated with each commit during
the git lifetime, then the full metadata will be present at the end ...
This I think is the crux of the discussion.
There are a number of alternate schemes that can be used to preserve the
1. (current FileTree implementation) store the Monticello meta data in git
and update on
every git commit
2. duplicate the existing Monticello revision history by committing in
order all of the
package ancestors and arrange for a way to reconstruct Monticello
metadata from git
meta data
3. (variant of 1) store the original monticello meta data in a file, do
not update on every commit
but arrange for a way to reconstruct Monticello metadata from git meta
data and graft onto
original Monticello meta data for use in mcz repository --- on demand
4. ????
Option 3 seems to be a good compromise solution and perhaps is the approach
that should be adopted moving forward ... we get to preserver Monticello
metadata while avoiding the messy commit conflicts for git while providing a
(somewhat) seamless path for a package to migrate back into the mcz
repository world ... if we somehow incorporate the SHA of the commit and the
github/bitbucket url into the revision history, then it would be possible to
perform a 3 way merge involving two mcz package versions and common ancestor
that is only present in git ...
Post by Dale Henrichs
Despite the fact that the Monticello metadata is redundant, I have made
sure that the Monticello metadata was included in FileTree from the very
beginning for the very reason that I wanted developers to be able to try out
FileTree, git and github without having to burn any Monticello bridges ....
if they didn't like FileTree, git and github, then they would be able to
back out of their use of git without losing data ...
Then forgive me. I couldn't see the wood for the trees. When I read your
talk of eliminating the conflicts from git commits due to the Monticello
metadata I infer that you're eliminating the Monticello metadata. I'm not
sure I understand the implications of this, but it seems to me that a
natural consequence is that the Montivcello metadata is lost and at that
point merges and the like become problematic. What am I missing?
You've hit the nail on the head, but I think that option 3 above gives us a
way to avoid losing the Monticello metadata without incurring a hit for a
packages lifetime while in git ...
A package that starts its life in git will have an empty Monticello metadata
and a package that never makes it's way into an mcz repository will not
incur per commit penalties ...
Post by Dale Henrichs
Post by Eliot Miranda
It seems entirely destructive.
It is not destructive ...
Post by Eliot Miranda
We have a functional package manager which currently supports interchange
between Pharo, Squeak and Cuis,
and GemStone?
I assume that you are talking about Monticello packages and Monticello
repositories ... or what?
Yes.
Post by Dale Henrichs
I am really not trying to do anything but "invent the future" --- I am not
trying to destroy, I am trying to improve ... If you are not able to see the
shortcomings of Monticello repositories (Note that I am distinguishing
between Monticello packages and Monticello repositories --- FileTree uses
Monticello packages and replaces Monticello repositories with git) and where
git has advantages over Monticello repositories, then you should continue to
use Monticello repositories ...
But what does this imply to some package that starts off in a Monticello
repository and then spends some time in gitland? Can I merge again? If I
can I'm happy. If I can't, I feel sabotaged.
and sabotage was never my intent ...
Post by Dale Henrichs
Personally I don't see Monticello repositories going away anytime soon and
expect to support Monticello repositories in GsDevKit_home, tODE, and
Metacello for the rest of my life:)
Fine. Except merging is, IIUC, about method time stamps and ancestry. If
that gets preserved then I'm happy. But for the life of me I haven't read
an explanation that reassures me that these are being preserved. Do you see
the roots of my fear?
Haha, from the very beginning back in 2012, I understood that there would be
fear and resistance to change and anger and joy and excitement but it was
not clear when if ever we'd reach point where a resolution for this
"problem" was needed: either no one would be interested or everyone would be
interested or ???
I really think that option 3 is going to be the best compromise moving
forward - there is some implementation work that will be required but I
really think option 3 gives you (and frankly me) a way to preserve the
Monticello revision history for packages that make their way back and forth
between lifetimes in git and Monticello repositories.
Eliot, I appreciate the fact that you demanded a better solution!
Dale
I'm curious how the merge driver is implemented. I think it was
mentioned that git calls-back to Pharo to do the processing. Is this
it something like this...

* How to make Git preserve specific files while merging
https://medium.com/@porteneuve/how-to-make-git-preserve-specific-files-while-merging-18c92343826b#.raovdvj9p

* A few of my Git tricks, tips and workflows
section under gitattribute(5)
http://nuclearsquid.com/writings/git-tricks-tips-workflows/


Or... rather than the metadata being a specific file (e.g.
'metadata'), the file could change for each commit (e.g.
'metadata.nnn') where nnn is the revision number, so each commit
deletes the old metadata file and writes the new metadata file.
Monticello could know to use whichever single 'metadata.*' file
exists. Would that avoid the problem of metadata merge conflict?

cheers -ben
Nicolas Cellier
2016-02-03 07:49:31 UTC
Permalink
https://github.com/ThierryGoubier/GitFileTree-MergeDriver
Post by Ben Coman
On Wed, Feb 3, 2016 at 7:00 AM, Dale Henrichs
Post by Christophe Demarey
Hi Dale,
On Tue, Feb 2, 2016 at 11:35 AM, Dale Henrichs
Post by Dale Henrichs
Post by Eliot Miranda
Hi David,
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is
starting
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method
format, even though that is relatively hostile to git user interfaces
optimised for other languages. There is really a need for a
file-per-class
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
exchange format, because that would works a lot better with the
existing VCS
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
ecosystem.
I agree so strongly. Class file outs which are eg sorted by selector
make much more sense. They won't hit the file name length limit.
They make
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
it trivial to maintain method and class comment time stamps. They're
easier
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
to construct into snapshots because it's easier to decode the file
name.
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
And then it's easy to add files for package load/unload scripts and for
the history. And then one is much more decoupled from the specific
back
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
end. It could be mercurial just as easily as git.
Post by David Allouche
I think more package-based user interfaces would indeed be a very good
idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular because it
is
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
a new shiny toy. I disagree with this idea. Git is a typical
worse-is-better
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
tool. It's good enough for most people, but it still has many
shortcomings.
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
It is popular in spite of its shortcomings. It became popular as
destination
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
for projects shifting from CVS and Subversion. So it is unlikely to be
displaced by a newer, incrementally shinier tools. Anything that will
displace it will have to provide an improvement of a similar
magnitude as
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
Post by David Allouche
the jump between centralised and distributed version control.
This is a good analysis. What's valuable to the Pharo community is not
displacing an already functional dvcs (Monticello) with an ill-suited
one
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
(git), but in being able to function in ecosystems like github where
people
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
can display their identity and where infrastructure for bug reports etc
exist.
Post by David Allouche
Still, I think it's a good idea not to restrict high level models to
what git provides if that's a less than ideal fit to the image model.
Absolutely. Dale's talk of ditching Monticello metadata fills me with
repulsion and makes me want to ask is he trying to sabotage or what?
Eliot, you can't be serious - accusing me of sabotage? Ah, well.... how
about you assume that I'm doing "or what":)
The Monticello metadata in a git repository is redundant and leads to
unnecessary commit conflicts -- end of story ....
No it's /not/ the end of the story. The essential part of the story is
how
Post by Christophe Demarey
Monticello remains compatible and interoperable between dialects. I
haven't
Post by Christophe Demarey
seen you account for how you maintain that compatibility. As far as I
can
Post by Christophe Demarey
tell, you propose replacing the Monticello metadata with that from git.
How
Post by Christophe Demarey
do I, as a Squeak user with Monticello, ever get to look at your package
again? As I understand it, moving the metadata from Monticello commit
time
Post by Christophe Demarey
to git means that the metadata is in a format that git determines, not
Monticello.
Good question, FileTree has been supported on Squeak since the very
beginning (I along with a small number of Squeak users have made sure of
that).
So TODAY, any Squeak user can "look at, load and commit" any package that
has been written using FileTree (with or without Monticello meta data).
[1] https://github.com/dalehenrich/filetree/tree/squeak4.3#squeak
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary commit
conflicts -- end of story ....", which implies you want to eliminate the
Monticello metadata, and on the other hand you say you're keeping the
Monticello metadata. I'm hopelessly confused. How does the Monticello
metadata get reconstituted if it's been thrown away?
Monticello meta data is not an integral part of the "package-ness" of a
Monticello package ... it _is_ integral to the "repository-ness" of a
Monticello package ...
If the Monticello metadata is "thrown away" then the revision history for
Monticello is lost, but for a package that is "born" in a git repository,
the Monticello metadata is not needed. Git has it's own commit meta data
and
Post by Christophe Demarey
the Monticello metadata is redundant.
If you want to see the revision history from a git-based FileTree repo,
1. one can include the Monticello meta data as part of the package -
this
Post by Christophe Demarey
is what FileTree
currently does
2. one can build a tool that reconstructs the Monticello version
history
Post by Christophe Demarey
from the git metadata
making it possible to use "old" Monitcello tools to look at the git
repo - I believe that Thierry's
GitFileTree takes this approach for metadata-less repositories
3. One can build a new tool that presents the git metadata without
reconstructing the
Monticello metadata at all. Note that by "embracing git", it is
possible to present revision
history at the package level (current mcz techology) as well as the
class and method level -
which is what I do in tODE
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
The above workflow can be accomplished whether or not Monticello
metadata is
Post by Christophe Demarey
present, however, if one does not make an effort to preserve the revision
history then at the end of your workflow the Monticello metadata is lost.
If one takes the pains to preserve the Monticello metadata before
committing
Post by Christophe Demarey
to the git repository and the metadata is updated with each commit during
the git lifetime, then the full metadata will be present at the end ...
This I think is the crux of the discussion.
There are a number of alternate schemes that can be used to preserve the
1. (current FileTree implementation) store the Monticello meta data in
git
Post by Christophe Demarey
and update on
every git commit
2. duplicate the existing Monticello revision history by committing in
order all of the
package ancestors and arrange for a way to reconstruct Monticello
metadata from git
meta data
3. (variant of 1) store the original monticello meta data in a file, do
not update on every commit
but arrange for a way to reconstruct Monticello metadata from git
meta
Post by Christophe Demarey
data and graft onto
original Monticello meta data for use in mcz repository --- on
demand
Post by Christophe Demarey
4. ????
Option 3 seems to be a good compromise solution and perhaps is the
approach
Post by Christophe Demarey
that should be adopted moving forward ... we get to preserver Monticello
metadata while avoiding the messy commit conflicts for git while
providing a
Post by Christophe Demarey
(somewhat) seamless path for a package to migrate back into the mcz
repository world ... if we somehow incorporate the SHA of the commit and
the
Post by Christophe Demarey
github/bitbucket url into the revision history, then it would be
possible to
Post by Christophe Demarey
perform a 3 way merge involving two mcz package versions and common
ancestor
Post by Christophe Demarey
that is only present in git ...
Post by Dale Henrichs
Despite the fact that the Monticello metadata is redundant, I have made
sure that the Monticello metadata was included in FileTree from the very
beginning for the very reason that I wanted developers to be able to
try out
Post by Christophe Demarey
Post by Dale Henrichs
FileTree, git and github without having to burn any Monticello bridges
....
Post by Christophe Demarey
Post by Dale Henrichs
if they didn't like FileTree, git and github, then they would be able to
back out of their use of git without losing data ...
Then forgive me. I couldn't see the wood for the trees. When I read
your
Post by Christophe Demarey
talk of eliminating the conflicts from git commits due to the Monticello
metadata I infer that you're eliminating the Monticello metadata. I'm
not
Post by Christophe Demarey
sure I understand the implications of this, but it seems to me that a
natural consequence is that the Montivcello metadata is lost and at that
point merges and the like become problematic. What am I missing?
You've hit the nail on the head, but I think that option 3 above gives
us a
Post by Christophe Demarey
way to avoid losing the Monticello metadata without incurring a hit for a
packages lifetime while in git ...
A package that starts its life in git will have an empty Monticello
metadata
Post by Christophe Demarey
and a package that never makes it's way into an mcz repository will not
incur per commit penalties ...
Post by Dale Henrichs
Post by Eliot Miranda
It seems entirely destructive.
It is not destructive ...
Post by Eliot Miranda
We have a functional package manager which currently supports
interchange
Post by Christophe Demarey
Post by Dale Henrichs
Post by Eliot Miranda
between Pharo, Squeak and Cuis,
and GemStone?
I assume that you are talking about Monticello packages and Monticello
repositories ... or what?
Yes.
Post by Dale Henrichs
I am really not trying to do anything but "invent the future" --- I am
not
Post by Christophe Demarey
Post by Dale Henrichs
trying to destroy, I am trying to improve ... If you are not able to
see the
Post by Christophe Demarey
Post by Dale Henrichs
shortcomings of Monticello repositories (Note that I am distinguishing
between Monticello packages and Monticello repositories --- FileTree
uses
Post by Christophe Demarey
Post by Dale Henrichs
Monticello packages and replaces Monticello repositories with git) and
where
Post by Christophe Demarey
Post by Dale Henrichs
git has advantages over Monticello repositories, then you should
continue to
Post by Christophe Demarey
Post by Dale Henrichs
use Monticello repositories ...
But what does this imply to some package that starts off in a Monticello
repository and then spends some time in gitland? Can I merge again? If
I
Post by Christophe Demarey
can I'm happy. If I can't, I feel sabotaged.
and sabotage was never my intent ...
Post by Dale Henrichs
Personally I don't see Monticello repositories going away anytime soon
and
Post by Christophe Demarey
Post by Dale Henrichs
expect to support Monticello repositories in GsDevKit_home, tODE, and
Metacello for the rest of my life:)
Fine. Except merging is, IIUC, about method time stamps and ancestry.
If
Post by Christophe Demarey
that gets preserved then I'm happy. But for the life of me I haven't
read
Post by Christophe Demarey
an explanation that reassures me that these are being preserved. Do you
see
Post by Christophe Demarey
the roots of my fear?
Haha, from the very beginning back in 2012, I understood that there
would be
Post by Christophe Demarey
fear and resistance to change and anger and joy and excitement but it was
not clear when if ever we'd reach point where a resolution for this
"problem" was needed: either no one would be interested or everyone
would be
Post by Christophe Demarey
interested or ???
I really think that option 3 is going to be the best compromise moving
forward - there is some implementation work that will be required but I
really think option 3 gives you (and frankly me) a way to preserve the
Monticello revision history for packages that make their way back and
forth
Post by Christophe Demarey
between lifetimes in git and Monticello repositories.
Eliot, I appreciate the fact that you demanded a better solution!
Dale
I'm curious how the merge driver is implemented. I think it was
mentioned that git calls-back to Pharo to do the processing. Is this
it something like this...
* How to make Git preserve specific files while merging
* A few of my Git tricks, tips and workflows
section under gitattribute(5)
http://nuclearsquid.com/writings/git-tricks-tips-workflows/
Or... rather than the metadata being a specific file (e.g.
'metadata'), the file could change for each commit (e.g.
'metadata.nnn') where nnn is the revision number, so each commit
deletes the old metadata file and writes the new metadata file.
Monticello could know to use whichever single 'metadata.*' file
exists. Would that avoid the problem of metadata merge conflict?
cheers -ben
Thierry Goubier
2016-02-03 09:13:02 UTC
Permalink
Hi Ben,

Le 03/02/2016 06:00, Ben Coman a écrit :>
Post by Ben Coman
I'm curious how the merge driver is implemented. I think it was
mentioned that git calls-back to Pharo to do the processing. Is this
it something like this...
* How to make Git preserve specific files while merging
Probably, haven't read it.
Post by Ben Coman
* A few of my Git tricks, tips and workflows
section under gitattribute(5)
http://nuclearsquid.com/writings/git-tricks-tips-workflows/
Yes. You use git attributes to trigger the use of a pharo-implemented
command line tool which does the merge for git.
Post by Ben Coman
Or... rather than the metadata being a specific file (e.g.
'metadata'), the file could change for each commit (e.g.
'metadata.nnn') where nnn is the revision number, so each commit
deletes the old metadata file and writes the new metadata file.
Monticello could know to use whichever single 'metadata.*' file
exists. Would that avoid the problem of metadata merge conflict?
I've thought of that, and it seems hard to get right (which file goes
first? how do you recognize a merge?). You have to remember that
packages have hundreds, even thousands of versions so this would mean
that number of files.

I went through all the different possible file formats, class-based,
package-based, method-based, log metadata and the like, and I concluded
that:

- the method based format is as good as any other.
- method based format allow for method-history queries on the git/vcs
history (as well as class based / package based queries).
- the tree structure on github or bitbucket is quite convenient (and
browsable) to the point one could edit a package directly in it (I do
when I need to do a quick fix).
- anything that can compress a bit the metadata version is probably good
to consider. version files can be huge.
- merge drivers really work and releave us from conflict resolution
- we need a merge tool written in Smalltalk/MC
- MC version numbering is a very bad idea
- MC almost never using properly UUIDs is a very bad behavior
- MC packages history can be considered broken in the general case
- it takes time to define, implement, test and really use a new format
and tooling

Thierry
Dale Henrichs
2016-02-03 17:01:19 UTC
Permalink
Thierry,

Very good points and I agree with all of them ...

Regarding a Smalltalk-based merge tool, I've written a git mergetool for
tODE (consider it a prototype) that could be adapted/ported for Pharo ....

My thoughts regarding the MC issues ... Fyou make good points some of
the "internal bugs" in MC and I think that it is important to point out
that the "internal bugs" you are referring to only affect Monticello
with regards to its repository functionality and not its package
functionality .

My inclination is to move forward and create a new, simpler package
model (Cypress is a good name) that is used with disk-based repos ...
the new package model would be protocol compatible with MC so that
definition-based snapshot comparisons continue. The new package model
would only be used for loading and there could/should be facilities for
converting between MC and this new package model.

FileTree would continue to be MC based and Cypress would be able to read
FileTree and FileTree would be able to read Cypress.

The improvements and changes that we are discussing (many of which
require a fair amount of implementation work) would be done in Cypress ...

The rationale is that Monticello packages encompass both the repository
and the code/definitions and for disk-based scms, Cypress would only
need to worry about the code/definitions --- and would make it possible
to move away from the version number-based package names that aren't
necessary when the underlying scm is doing the versioning without
affecting the MC implementation ....

Of course, if a new Cypress package model were introduced, the tools
would need to change, but since we are already considering changing
tools, this would be the perfect time to improve the underlying model
for disk-based SCMs.

Dale
Post by Thierry Goubier
I went through all the different possible file formats, class-based,
package-based, method-based, log metadata and the like, and I
- the method based format is as good as any other.
- method based format allow for method-history queries on the git/vcs
history (as well as class based / package based queries).
- the tree structure on github or bitbucket is quite convenient (and
browsable) to the point one could edit a package directly in it (I do
when I need to do a quick fix).
- anything that can compress a bit the metadata version is probably
good to consider. version files can be huge.
- merge drivers really work and releave us from conflict resolution
- we need a merge tool written in Smalltalk/MC
- MC version numbering is a very bad idea
- MC almost never using properly UUIDs is a very bad behavior
- MC packages history can be considered broken in the general case
- it takes time to define, implement, test and really use a new format
and tooling
Thierry Goubier
2016-02-03 09:18:29 UTC
Permalink
Hi Ben,

Le 03/02/2016 06:00, Ben Coman a écrit :>
Post by Ben Coman
I'm curious how the merge driver is implemented. I think it was
mentioned that git calls-back to Pharo to do the processing. Is this
it something like this...
* How to make Git preserve specific files while merging
Probably, haven't read it.
Post by Ben Coman
* A few of my Git tricks, tips and workflows
section under gitattribute(5)
http://nuclearsquid.com/writings/git-tricks-tips-workflows/
Yes. You use git attributes to trigger the use of a pharo-implemented
command line tool which does the merge for git.
Post by Ben Coman
Or... rather than the metadata being a specific file (e.g.
'metadata'), the file could change for each commit (e.g.
'metadata.nnn') where nnn is the revision number, so each commit
deletes the old metadata file and writes the new metadata file.
Monticello could know to use whichever single 'metadata.*' file
exists. Would that avoid the problem of metadata merge conflict?
I've thought of that, and it seems hard to get right (which file goes
first? how do you recognize a merge?). You have to remember that
packages have hundreds, even thousands of versions so this would mean
that number of files.

I went through all the different possible file formats, class-based,
package-based, method-based, log metadata and the like, and I concluded
that:

- the method based format is as good as any other. Even better since it
has a spec (cypress).
- method based format allow for method-history queries on the git/vcs
history (as well as class based / package based queries).
- the tree structure on github or bitbucket is quite convenient (and
browsable) to the point one could edit a package directly in it (I do
when I need to do a quick fix).
- anything that can compress a bit the metadata version is probably good
to consider. version files can be huge.
- merge drivers really work and releave us from conflict resolution
- we need a merge tool written in Smalltalk/MC
- MC version numbering is a very bad idea
- MC almost never using properly UUIDs is a very bad behavior
- MC packages history can be considered broken in the general case
- it takes time to define, implement, test and really use a new format
and tooling

Thierry
Sven Van Caekenberghe
2016-02-03 09:44:25 UTC
Permalink
Thanks Thierry for pushing this subject and working so patiently with the community.
- the method based format is as good as any other. Even better since it has a spec (cypress).
- method based format allow for method-history queries on the git/vcs history (as well as class based / package based queries).
- the tree structure on github or bitbucket is quite convenient (and browsable) to the point one could edit a package directly in it (I do when I need to do a quick fix).
- anything that can compress a bit the metadata version is probably good to consider. version files can be huge.
- merge drivers really work and releave us from conflict resolution
- we need a merge tool written in Smalltalk/MC
- MC version numbering is a very bad idea
- MC almost never using properly UUIDs is a very bad behavior
- MC packages history can be considered broken in the general case
- it takes time to define, implement, test and really use a new format and tooling
Excellent summary.

I also feel that the current structure (basic filetree) is more than OK. You can /almost/ browse through github.

I am sure we'll get there with the meta data and tooling.

Sven
Thierry Goubier
2016-02-03 08:54:27 UTC
Permalink
Hi Eliot,

Le 02/02/2016 21:54, Eliot Miranda a écrit :
....
Post by Eliot Miranda
No it's /not/ the end of the story. The essential part of the story is
how Monticello remains compatible and interoperable between dialects. I
haven't seen you account for how you maintain that compatibility. As
far as I can tell, you propose replacing the Monticello metadata with
that from git. How do I, as a Squeak user with Monticello, ever get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
Post by Eliot Miranda
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say you're
keeping the Monticello metadata. I'm hopelessly confused. How does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.

The only thing that is problematic there is that the only safe history
is the one generated from git... there are so many MC packages with
broken history that, on mcz packages, you have to admit that it's not
safe to base things on their history.
Post by Eliot Miranda
But what does this imply to some package that starts off in a Monticello
repository and then spends some time in gitland? Can I merge again? If
I can I'm happy. If I can't, I feel sabotaged.
You could. Just express your needs and wait until one of us has enough
free time to solve it for you, that is.
Post by Eliot Miranda
Fine. Except merging is, IIUC, about method time stamps and ancestry.
If that gets preserved then I'm happy. But for the life of me I haven't
read an explanation that reassures me that these are being preserved.
Do you see the roots of my fear?
Of course. But that also project a bit what you think of the people
working on it... which may makes it a bit hard to answer.

Thierry
Eliot Miranda
2016-02-03 21:51:11 UTC
Permalink
Post by Thierry Goubier
Hi Eliot,
....
Post by Eliot Miranda
No it's /not/ the end of the story. The essential part of the story is
how Monticello remains compatible and interoperable between dialects. I
haven't seen you account for how you maintain that compatibility. As
far as I can tell, you propose replacing the Monticello metadata with
that from git. How do I, as a Squeak user with Monticello, ever get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The Monticello
Post by Eliot Miranda
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say you're
keeping the Monticello metadata. I'm hopelessly confused. How does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe history is
the one generated from git... there are so many MC packages with broken
history that, on mcz packages, you have to admit that it's not safe to base
things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history in
our mczs that is correct. Certainly in VMMaker.oscog I have history that
goes back a long time. If bugs have broken history then efforts should be
made to repair that history. But you can't just write off Monticello
history like that.

But what does this imply to some package that starts off in a Monticello
Post by Thierry Goubier
Post by Eliot Miranda
repository and then spends some time in gitland? Can I merge again? If
I can I'm happy. If I can't, I feel sabotaged.
You could. Just express your needs and wait until one of us has enough
free time to solve it for you, that is.
Fine. Except merging is, IIUC, about method time stamps and ancestry.
Post by Eliot Miranda
If that gets preserved then I'm happy. But for the life of me I haven't
read an explanation that reassures me that these are being preserved.
Do you see the roots of my fear?
Of course. But that also project a bit what you think of the people
working on it... which may makes it a bit hard to answer.
Thierry
_,,,^..^,,,_
best, Eliot
Thierry Goubier
2016-02-03 22:34:52 UTC
Permalink
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the story is
how Monticello remains compatible and interoperable between dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello metadata with
that from git. How do I, as a Squeak user with Monticello, ever get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say you're
keeping the Monticello metadata. I'm hopelessly confused. How does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.

I write tools that work with Monticello repositories, not just yours. I
have to do with what is given to me. On a general level, as a mcz user,
I'll just have to consider that you are as susceptible to be trusted as
with any other mcz producer. This means not much...

Thierry
Dale Henrichs
2016-02-03 22:58:40 UTC
Permalink
Post by Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the story is
how Monticello remains compatible and interoperable between dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello metadata with
that from git. How do I, as a Squeak user with Monticello, ever get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say you're
keeping the Monticello metadata. I'm hopelessly confused. How does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....

Correct or not, the Monticello version history should be preserved ....

Dale
Thierry Goubier
2016-02-04 06:53:38 UTC
Permalink
Post by Dale Henrichs
Post by Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the story is
how Monticello remains compatible and interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello metadata with
that from git. How do I, as a Squeak user with Monticello, ever get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say you're
keeping the Monticello metadata. I'm hopelessly confused. How
does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is implemented.
And I agree this may well be the way to go. But ...
Post by Dale Henrichs
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history is
that it has value when you can access the versions listed in it. If you
mix repositories like that, unless you maintain a link to the previous
mcz repository, pre-git versions can't be accessed. So, most of the
time, what we do with a project moved under git is to clone the previous
repository, not take just the current head.

(you'll notice, by the way, that vcs usually work that way when moving
from, say, CVS to git - you move the entire repository, not just the
latest version).

In short, the question would really be:
1- should we invest into making that integration of past history a
selling point (but I foresee issues down the road; I've only described
one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?

Honestly, I'd consider 1- to be the easiest to implement. 2- there is
already some code floating around.

Thierry
Nicolas Cellier
2016-02-04 09:04:21 UTC
Permalink
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together in
metadata no?
If some tool only trust .mcz name without checking UUID, consider it's a
bug, and let's correct it (there is some in Monticello Configuration Map)

Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's just
that you'll have to redo the merge if you lost a common ancestor).

Or is it the fact that some repository might contain only a slice of
history?
This is another feature... You can view all the versions in a collection of
repositories without needing to replicate.
You can replicate if you want but it's not mandatory and completely
orthogonal.
So yes, this information - the list or repositories you want to consider -
has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very low
investment.
And that also mean that you can hardly break things (have unconsistent
history).

Maybe when you say broken, you mean not 100% git compatible?
Post by Thierry Goubier
Post by Dale Henrichs
Post by Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the
story is
how Monticello remains compatible and interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello metadata with
that from git. How do I, as a Squeak user with Monticello, ever
get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you say
you're
keeping the Monticello metadata. I'm hopelessly confused. How
does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is implemented.
And I agree this may well be the way to go. But ...
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history is
that it has value when you can access the versions listed in it. If you mix
repositories like that, unless you maintain a link to the previous mcz
repository, pre-git versions can't be accessed. So, most of the time, what
we do with a project moved under git is to clone the previous repository,
not take just the current head.
(you'll notice, by the way, that vcs usually work that way when moving
from, say, CVS to git - you move the entire repository, not just the latest
version).
1- should we invest into making that integration of past history a selling
point (but I foresee issues down the road; I've only described one so far,
and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there is
already some code floating around.
Thierry
Thierry Goubier
2016-02-04 09:26:49 UTC
Permalink
Post by Nicolas Cellier
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together
in metadata no?
Yes.
Post by Nicolas Cellier
If some tool only trust .mcz name without checking UUID, consider it's a
bug, and let's correct it (there is some in Monticello Configuration Map)
Gofer. In fact, most of Monticello never checks the UUID, only
dependencies do, but this looks like a deprecated feature given how
often it is used.
Post by Nicolas Cellier
Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's
just that you'll have to redo the merge if you lost a common ancestor).
What? You really consider that a feature?
Post by Nicolas Cellier
Or is it the fact that some repository might contain only a slice of
history?
This is another feature... You can view all the versions in a collection
of repositories without needing to replicate.
Understandable in theory. Unworkable over time and change (repositories
disappear and die, and this stops working)
Post by Nicolas Cellier
You can replicate if you want but it's not mandatory and completely
orthogonal.
So yes, this information - the list or repositories you want to consider
- has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very
low investment.
And that also mean that you can hardly break things (have unconsistent
history).
Maybe when you say broken, you mean not 100% git compatible?
No, what I say is true mcz inconsistent history (missing versions making
merges very unreliable, basically).

I describe a while ago a case where, thanks to mcz features, I couldn't
merge a small change done to Roassal without generating a ton of
conflicts. I moved the three needed mcz(s) to git (the change ancestor,
the change, and the current head), did git merge and had no conflicts.

If you consider those features, then I disagree.

I'd really like to improve MC and get a better integration. Considering
those as features just make me think that Eliot may well be right in
believing we'll end up throwing away MC completely.

Coping with MC idiosyncrasies is hard as it is, and at least a complete
replacement is being considered for Pharo. In the meantime, Cuis has
completely given up managing packages in Smalltalk: all is done in git,
externally.

Thierry
Post by Nicolas Cellier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The
essential part of the
story is
how Monticello remains compatible and
interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the
Monticello metadata
with
that from git. How do I, as a Squeak user with
Monticello, ever
get to
look at your package again? As I understand
it, moving the
metadata
from Monticello commit time to git means that
the metadata is
in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you
can say "The
Monticello
metadata in a git repository is redundant and
leads to
unnecessary
commit conflicts -- end of story ....", which
implies you
want to
eliminate the Monticello metadata, and on the
other hand you say
you're
keeping the Monticello metadata. I'm
hopelessly confused. How
does the
Monticello metadata get reconstituted if it's
been thrown away?
What happens to the metadata in the following
workflow?
load package P from Monticello repository R
into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on,
but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that
the only safe
history is the one generated from git... there are
so many MC
packages with broken history that, on mcz packages,
you have to
admit that it's not safe to base things on their
history.
I'm sorry but I don't accept that. In the Squeak trunk
we have history
in our mczs that is correct. Certainly in VMMaker.oscog
I have history
that goes back a long time. If bugs have broken history
then efforts
should be made to repair that history. But you can't
just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not
just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is
implemented. And I agree this may well be the way to go. But ...
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history
is that it has value when you can access the versions listed in it.
If you mix repositories like that, unless you maintain a link to the
previous mcz repository, pre-git versions can't be accessed. So,
most of the time, what we do with a project moved under git is to
clone the previous repository, not take just the current head.
(you'll notice, by the way, that vcs usually work that way when
moving from, say, CVS to git - you move the entire repository, not
just the latest version).
1- should we invest into making that integration of past history a
selling point (but I foresee issues down the road; I've only
described one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there
is already some code floating around.
Thierry
Henrik Johansen
2016-02-04 12:09:37 UTC
Permalink
Post by Nicolas Cellier
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together
in metadata no?
Yes.
Post by Nicolas Cellier
If some tool only trust .mcz name without checking UUID, consider it's a
bug, and let's correct it (there is some in Monticello Configuration Map)
Gofer. In fact, most of Monticello never checks the UUID, only dependencies do, but this looks like a deprecated feature given how often it is used.
Post by Nicolas Cellier
Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's
just that you'll have to redo the merge if you lost a common ancestor).
What? You really consider that a feature?
Post by Nicolas Cellier
Or is it the fact that some repository might contain only a slice of
history?
This is another feature... You can view all the versions in a collection
of repositories without needing to replicate.
Understandable in theory. Unworkable over time and change (repositories disappear and die, and this stops working)
Post by Nicolas Cellier
You can replicate if you want but it's not mandatory and completely
orthogonal.
So yes, this information - the list or repositories you want to consider
- has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very
low investment.
And that also mean that you can hardly break things (have unconsistent
history).
Maybe when you say broken, you mean not 100% git compatible?
No, what I say is true mcz inconsistent history (missing versions making merges very unreliable, basically).
I describe a while ago a case where, thanks to mcz features, I couldn't merge a small change done to Roassal without generating a ton of conflicts. I moved the three needed mcz(s) to git (the change ancestor, the change, and the current head), did git merge and had no conflicts.
If you consider those features, then I disagree.
Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*.
I don't think anyone looked into why it happened.

Cheers,
Henry

*http://forum.world.st/FogBugz-Case-Issue-12776-Tools-Cancelling-quot-Previous-Contents-quot-in-a-Workspace-clears-the-currs-td4753381.html
Stephan Eggermont
2016-02-04 12:21:40 UTC
Permalink
Post by Henrik Johansen
Also, the ancestry of all packages in Pharo Core was truncated a few years back, which also screwed up merging*.
I don't think anyone looked into why it happened.
My experiment in creating one file for a whole project would be a
suitable starting point for fixing that.

Stephan
Thierry Goubier
2016-02-04 13:56:04 UTC
Permalink
Post by Stephan Eggermont
Post by Henrik Johansen
Also, the ancestry of all packages in Pharo Core was truncated a few
years back, which also screwed up merging*.
I don't think anyone looked into why it happened.
My experiment in creating one file for a whole project would be a
suitable starting point for fixing that.
Yes. I think we need a MC version history recreation / garbage collect.
Something that removes dead versions and compress history to make MC
merge work correctly.

(unless you move to git on metadataless mode and back...).

Thierry
Post by Stephan Eggermont
Stephan
Dale Henrichs
2016-02-04 16:22:13 UTC
Permalink
Stephan,

Have you described this anywhere?

Dale
Post by Stephan Eggermont
Post by Henrik Johansen
Also, the ancestry of all packages in Pharo Core was truncated a few
years back, which also screwed up merging*.
I don't think anyone looked into why it happened.
My experiment in creating one file for a whole project would be a
suitable starting point for fixing that.
Stephan
Stephan Eggermont
2016-02-04 17:28:04 UTC
Permalink
Post by Dale Henrichs
Stephan,
Have you described this anywhere?
Not very much

http://forum.world.st/Overhead-in-mczs-MCVersionInfo-td4869374.html

MCHttpRepository
location:
'http://smalltalkhub.com/mc/StephanEggermont/MonticelloProjects/main'
user: ''
password: ''

Jan 5. on Slack

Stephan Eggermont
20:21 So I deduplicated the ancestry, putting the UUID of missing
ancestors in place of the MCVersionInfo
20:25
123 MB instead of 1.2 GB, compressed 70.8 MB
20:26
The whole history of VMMaker
Thierry Goubier
20:34 With all versions?
Stephan Eggermont
20:34 missing 2,
20:35
and only those on source.squeak.org
Stephan Eggermont
20:35
CompactVMMaker.fl.tar.gz

67MB Binary Click to download
Thierry Goubier
20:37 I'd be interested to see how large would be the git repository
containing that. Maybe I'll make a try.
Stephan Eggermont
20:42 Nice compression factor. package cache of 2725 items, 6.1 GB
Thierry Goubier
20:44 !
Thierry Goubier
21:21 Then I can't test the git thing. Not enough HD space on the
Chromebook.
Thierry Goubier
2016-02-04 13:22:36 UTC
Permalink
Post by Henrik Johansen
Also, the ancestry of all packages in Pharo Core was truncated a few
years back, which also screwed up merging*. I don't think anyone
looked into why it happened.
It was done on purpose for the Pharo 3 release to reduce the amount of
memory used by the MCVersionInfo instances.

For Pharo 4, a lazy loading of the MCVersionInfo was implemented
instead. We should have recreated the lost history then, but it seemed
not to have been important.

Thierry
Nicolas Cellier
2016-02-04 21:07:41 UTC
Permalink
Post by Thierry Goubier
Post by Nicolas Cellier
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together
in metadata no?
Yes.
Post by Nicolas Cellier
If some tool only trust .mcz name without checking UUID, consider it's a
bug, and let's correct it (there is some in Monticello Configuration
Map)
Post by Thierry Goubier
Gofer. In fact, most of Monticello never checks the UUID, only
dependencies do, but this looks like a deprecated feature given how often
it is used.
Post by Thierry Goubier
Post by Nicolas Cellier
Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's
just that you'll have to redo the merge if you lost a common ancestor).
What? You really consider that a feature?
Post by Nicolas Cellier
Or is it the fact that some repository might contain only a slice of
history?
This is another feature... You can view all the versions in a collection
of repositories without needing to replicate.
Understandable in theory. Unworkable over time and change (repositories
disappear and die, and this stops working)
Post by Thierry Goubier
Post by Nicolas Cellier
You can replicate if you want but it's not mandatory and completely
orthogonal.
So yes, this information - the list or repositories you want to consider
- has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very
low investment.
And that also mean that you can hardly break things (have unconsistent
history).
Maybe when you say broken, you mean not 100% git compatible?
No, what I say is true mcz inconsistent history (missing versions making
merges very unreliable, basically).
Post by Thierry Goubier
I describe a while ago a case where, thanks to mcz features, I couldn't
merge a small change done to Roassal without generating a ton of conflicts.
I moved the three needed mcz(s) to git (the change ancestor, the change,
and the current head), did git merge and had no conflicts.
Post by Thierry Goubier
If you consider those features, then I disagree.
Also, the ancestry of all packages in Pharo Core was truncated a few years
back, which also screwed up merging*.
I don't think anyone looked into why it happened.
Cheers,
Henry
*
http://forum.world.st/FogBugz-Case-Issue-12776-Tools-Cancelling-quot-Previous-Contents-quot-in-a-Workspace-clears-the-currs-td4753381.html
Henry, you perfectly nailed it...
Break it first/throw away/replace is not allways a winning strategy.
We are ready to re-integrate source code in image at the price of a few
MBytes, but it was urgent to throw history away BEFORE we even get a
working replacement...
Here is a mail i did not send because I didn't want to be to start the year
too negativily:

----------------------------

"snip... was about a regexp bug that I tried to bisect... (a post of
january 2015)
So I wanted to be informed of the other changes going on since plenty of
interesting things are going on in Pharo.

Ah hem, but the package has been renamed and the 'version history' menu is
there only to show me that history has been purposely expunged from
image... It became such a useless menu now, that i think you could as well
remove it.

Spawning the 4.0 repository and using the History button, reveals a bit of
history this time, sounds like the 3.0 bits...
Notice in attachment how the versions 22 and 23 which are in the 3.0
repository seems to be missing...
Ah, no this is a brand new strategy for presenting the merged branches...
without any care for a common ancestor.
Notice in other attachment how it's not allways been so.

And what if I want to look a bit further behind?
Shall I go outside the image and browse github now?
The web UI did absolutely not work for me, the 3.0 history is cut there
too, and blame only reveals me the name of Jenkins...
Git command line might have done a better work, but anyway, doesn't the I
in IDE stands for Integrated?
I won't know because at this stage i gave up.

snip... that's enough ranting "

Ben Coman
2016-02-04 12:22:47 UTC
Permalink
On Thu, Feb 4, 2016 at 5:26 PM, Thierry Goubier
In the meantime, Cuis has completely given up managing packages in
Smalltalk: all is done in git externally.
Thats very interesting to hear - since 2012 even...
http://www.jvuletich.org/Cuis/CuisAndGitHub.html

and interesting to compare different approaches...
* https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev/blob/master/Documentation/CuisAndGitHub.md

* https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev/blob/master/Documentation/CodeManagementInCuis.md

cheers -ben
Nicolas Cellier
2016-02-04 13:32:51 UTC
Permalink
Post by Nicolas Cellier
Post by Nicolas Cellier
I don't understand broken history either.
Yes there can be .mcz name clashes but UUID history is stored together
in metadata no?
Yes.
If some tool only trust .mcz name without checking UUID, consider it's a
Post by Nicolas Cellier
bug, and let's correct it (there is some in Monticello Configuration Map)
Gofer. In fact, most of Monticello never checks the UUID, only
dependencies do, but this looks like a deprecated feature given how often
it is used.
Or is it the fact that some .mcz could be missing?
Post by Nicolas Cellier
Consider this is a feature, MC tools are robust to missing .mcz (it's
just that you'll have to redo the merge if you lost a common ancestor).
What? You really consider that a feature?
Sure. Consider it's somehow like a git squash...
Post by Nicolas Cellier
Or is it the fact that some repository might contain only a slice of
Post by Nicolas Cellier
history?
This is another feature... You can view all the versions in a collection
of repositories without needing to replicate.
Understandable in theory. Unworkable over time and change (repositories
disappear and die, and this stops working)
But that's the same with over vcs, repositories disappear and die (google
code etc...).
Knowing that you can take usual defensive decisions (replicate what you
depend on and not trust too much).
Post by Nicolas Cellier
You can replicate if you want but it's not mandatory and completely
Post by Nicolas Cellier
orthogonal.
So yes, this information - the list or repositories you want to consider
- has to be stored separetely and this can sound quite unconventional.
But IMO, it's an important feature: it gives much resilience for a very
low investment.
And that also mean that you can hardly break things (have unconsistent
history).
Maybe when you say broken, you mean not 100% git compatible?
No, what I say is true mcz inconsistent history (missing versions making
merges very unreliable, basically).
Loosing a .mcz on which you branched is a bit more than unfortunate, it's
a lack of understanding of the tool...
I say give git squash and rebase to unexperimented/uninformed people and
that will be far worse.
Post by Nicolas Cellier
I describe a while ago a case where, thanks to mcz features, I couldn't
merge a small change done to Roassal without generating a ton of conflicts.
I moved the three needed mcz(s) to git (the change ancestor, the change,
and the current head), did git merge and had no conflicts.
Do you mean that git succeeded in merging text because different lines were
changed in different branches?
I agree that it can solve some basic refactoring (renaming...)
But couldn't it lead to syntactically incorrect code? Traditional Smalltalk
IDE are not well equipped for handling this...

The biggest grief I have with MC is more about package delimitation, when
code is moved from 1 package to another.
For example, this effectively prevent merging very distant branches like
Squeak and Pharo
(but we do not want to merge, just cherry pick in fact).
Post by Nicolas Cellier
If you consider those features, then I disagree.
I'd really like to improve MC and get a better integration. Considering
those as features just make me think that Eliot may well be right in
believing we'll end up throwing away MC completely.
As long as it's as efficient, and well integrated in IDE, but again, it's
against the small steps approach...
For me small steps would be to replace mcz name indexing by UUID indexing
and just keep name as a façade.
I'm quite sure most work is at server side. How difficult would it really
be?
No one will consider this maybe, because it's against the stream ;)
Post by Nicolas Cellier
Coping with MC idiosyncrasies is hard as it is, and at least a complete
replacement is being considered for Pharo. In the meantime, Cuis has
completely given up managing packages in Smalltalk: all is done in git,
externally.
But Cuis vcs come from dinosaure era (change sets) so anything else is
already a progress.
Doing that in Squeak would feel like a loss of
tool/functionality/integration.
Why didn't Pharo took the same path?

Thierry
Post by Nicolas Cellier
Post by Nicolas Cellier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The
essential part of the
story is
how Monticello remains compatible and
interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the
Monticello metadata
with
that from git. How do I, as a Squeak user with
Monticello, ever
get to
look at your package again? As I understand
it, moving the
metadata
from Monticello commit time to git means that
the metadata is
in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you
can say "The
Monticello
metadata in a git repository is redundant and
leads to
unnecessary
commit conflicts -- end of story ....", which
implies you
want to
eliminate the Monticello metadata, and on the
other hand you say
you're
keeping the Monticello metadata. I'm
hopelessly confused. How
does the
Monticello metadata get reconstituted if it's
been thrown away?
What happens to the metadata in the following
workflow?
load package P from Monticello repository R
into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on,
but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that
the only safe
history is the one generated from git... there are
so many MC
packages with broken history that, on mcz packages,
you have to
admit that it's not safe to base things on their
history.
I'm sorry but I don't accept that. In the Squeak trunk
we have history
in our mczs that is correct. Certainly in VMMaker.oscog
I have history
that goes back a long time. If bugs have broken history
then efforts
should be made to repair that history. But you can't
just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not
just yours.
I have to do with what is given to me. On a general level,
as a mcz
user, I'll just have to consider that you are as susceptible
to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is
implemented. And I agree this may well be the way to go. But ...
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history
is that it has value when you can access the versions listed in it.
If you mix repositories like that, unless you maintain a link to the
previous mcz repository, pre-git versions can't be accessed. So,
most of the time, what we do with a project moved under git is to
clone the previous repository, not take just the current head.
(you'll notice, by the way, that vcs usually work that way when
moving from, say, CVS to git - you move the entire repository, not
just the latest version).
1- should we invest into making that integration of past history a
selling point (but I foresee issues down the road; I've only
described one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there
is already some code floating around.
Thierry
Thierry Goubier
2016-02-04 13:52:52 UTC
Permalink
[... cut ]
Post by Nicolas Cellier
Or is it the fact that some .mcz could be missing?
Consider this is a feature, MC tools are robust to missing .mcz (it's
just that you'll have to redo the merge if you lost a common ancestor).
What? You really consider that a feature?
Sure. Consider it's somehow like a git squash...
I consider history rewriting a very bad idea...

But, if we are at that, I just consider that the removed versions do not
exist anymore, and are not visible (should not be visible) in the
history, and that you have perfectly predictable correct behavior, not
can't do this because package version X is missing.
Post by Nicolas Cellier
Or is it the fact that some repository might contain only a slice of
history?
This is another feature... You can view all the versions in a collection
of repositories without needing to replicate.
Understandable in theory. Unworkable over time and change
(repositories disappear and die, and this stops working)
But that's the same with over vcs, repositories disappear and die
(google code etc...).
Knowing that you can take usual defensive decisions (replicate what you
depend on and not trust too much).
I mean that VCS recreate a working history and log when you are in such
a situation. MC just let you believe you have all and fail when you
attain the holes.
Post by Nicolas Cellier
You can replicate if you want but it's not mandatory and completely
orthogonal.
So yes, this information - the list or repositories you want to consider
- has to be stored separetely and this can sound quite
unconventional.
But IMO, it's an important feature: it gives much resilience for a very
low investment.
And that also mean that you can hardly break things (have unconsistent
history).
Maybe when you say broken, you mean not 100% git compatible?
No, what I say is true mcz inconsistent history (missing versions
making merges very unreliable, basically).
Loosing a .mcz on which you branched is a bit more than unfortunate,
it's a lack of understanding of the tool...
Not true. MC let you believe it will cope and it doesn't, which is not
exactly the same.
Post by Nicolas Cellier
I say give git squash and rebase to unexperimented/uninformed people and
that will be far worse.
Agreed.
Post by Nicolas Cellier
I describe a while ago a case where, thanks to mcz features, I
couldn't merge a small change done to Roassal without generating a
ton of conflicts. I moved the three needed mcz(s) to git (the change
ancestor, the change, and the current head), did git merge and had
no conflicts.
Do you mean that git succeeded in merging text because different lines
were changed in different branches?
No, in fact it was a clean merge because the changes in the branch were
on methods untouched in the main... git recreated a proper history and
could merge, whereas MC with the mcz history generated conflicts without
any reason.
Post by Nicolas Cellier
I agree that it can solve some basic refactoring (renaming...)
But couldn't it lead to syntactically incorrect code? Traditional
Smalltalk IDE are not well equipped for handling this...
Yes, this is partly why we're looking at more integration (merge tool)
and we may have to do more (a smalltalk syntax checker before commit
could be nice; CI does wonders on that).
Post by Nicolas Cellier
The biggest grief I have with MC is more about package delimitation,
when code is moved from 1 package to another.
For example, this effectively prevent merging very distant branches like
Squeak and Pharo
(but we do not want to merge, just cherry pick in fact).
Ok. Merge with multiple packages, right?
Post by Nicolas Cellier
If you consider those features, then I disagree.
I'd really like to improve MC and get a better integration.
Considering those as features just make me think that Eliot may well
be right in believing we'll end up throwing away MC completely.
As long as it's as efficient, and well integrated in IDE, but again,
it's against the small steps approach...
For me small steps would be to replace mcz name indexing by UUID
indexing and just keep name as a façade.
I'd agree with that, as long as you don't ask too much of
compatibilities with some of MC features (:)).
Post by Nicolas Cellier
I'm quite sure most work is at server side. How difficult would it
really be?
Why server side? I don't think it is.
Post by Nicolas Cellier
No one will consider this maybe, because it's against the stream ;)
No, in fact we're on it. Git integration is showing us where MC is weak,
and we're trying to fit it with MC.
Post by Nicolas Cellier
Coping with MC idiosyncrasies is hard as it is, and at least a
complete replacement is being considered for Pharo. In the meantime,
Cuis has completely given up managing packages in Smalltalk: all is
done in git, externally.
But Cuis vcs come from dinosaure era (change sets) so anything else is
already a progress.
Doing that in Squeak would feel like a loss of
tool/functionality/integration.
I agree.
Post by Nicolas Cellier
Why didn't Pharo took the same path?
Than Cuis?

Thierry
Post by Nicolas Cellier
Thierry
2016-02-04 7:53 GMT+01:00 Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The
essential part of the
story is
how Monticello remains compatible and
interoperable between
dialects. I
haven't seen you account for how you
maintain that
compatibility. As
far as I can tell, you propose replacing the
Monticello metadata
with
that from git. How do I, as a Squeak user with
Monticello, ever
get to
look at your package again? As I understand
it, moving the
metadata
from Monticello commit time to git means that
the metadata is
in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you
can say "The
Monticello
metadata in a git repository is redundant and
leads to
unnecessary
commit conflicts -- end of story ....", which
implies you
want to
eliminate the Monticello metadata, and on the
other hand you say
you're
keeping the Monticello metadata. I'm
hopelessly confused. How
does the
Monticello metadata get reconstituted if it's
been thrown away?
What happens to the metadata in the following
workflow?
load package P from Monticello repository R
into an image
change P, commit via git to local git
repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on,
but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that
the only safe
history is the one generated from git... there are
so many MC
packages with broken history that, on mcz packages,
you have to
admit that it's not safe to base things on their
history.
I'm sorry but I don't accept that. In the Squeak trunk
we have history
in our mczs that is correct. Certainly in VMMaker.oscog
I have history
that goes back a long time. If bugs have broken history
then efforts
should be made to repair that history. But you can't
just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello
repositories, not
just yours.
I have to do with what is given to me. On a general level,
as a mcz
user, I'll just have to consider that you are as susceptible
to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor
option 3,
which
records the package version history as it existed at
the point
it was
copied into a git repo. When copied back out from the
git universe,
create a version history that starts with the original
version
history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is
implemented. And I agree this may well be the way to go. But ...
Correct or not, the Monticello version history should be
preserved ....
I wonder about that. The property of the Monticello version history
is that it has value when you can access the versions listed in it.
If you mix repositories like that, unless you maintain a link to the
previous mcz repository, pre-git versions can't be accessed. So,
most of the time, what we do with a project moved under git is to
clone the previous repository, not take just the current head.
(you'll notice, by the way, that vcs usually work that way when
moving from, say, CVS to git - you move the entire repository, not
just the latest version).
1- should we invest into making that integration of past history a
selling point (but I foresee issues down the road; I've only
described one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there
is already some code floating around.
Thierry
Dale Henrichs
2016-02-04 17:08:24 UTC
Permalink
Post by Thierry Goubier
Post by Dale Henrichs
Post by Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the
story is
how Monticello remains compatible and interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello
metadata
with
that from git. How do I, as a Squeak user with Monticello,
ever
get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand
you say
you're
keeping the Monticello metadata. I'm hopelessly confused.
How
does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is
implemented. And I agree this may well be the way to go. But ...
Post by Dale Henrichs
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history is
that it has value when you can access the versions listed in it. If
you mix repositories like that, unless you maintain a link to the
previous mcz repository, pre-git versions can't be accessed. So, most
of the time, what we do with a project moved under git is to clone the
previous repository, not take just the current head.
(you'll notice, by the way, that vcs usually work that way when moving
from, say, CVS to git - you move the entire repository, not just the
latest version).
1- should we invest into making that integration of past history a
selling point (but I foresee issues down the road; I've only described
one so far, and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there is
already some code floating around.
Thierry,

My take on this is that we are not trying to "fix" Monticello.

The developers who prefer to use Monticello should continue to use
Monticello and the existing Monticello tools should continue to be
available. Any new formats should provide a migration path from
Monticello and any new tools like a `project list` tool should
accommodate projects based on both git and Monticello repositories.

This particular branch of the conversation has been aimed at trying to
resolve the tension that has been created by including Monticello
metadata in FileTree repositories and I think that "option 3": including
past history in the FileTree repository, but not updating the version
history for each git commit is a workable compromise --- there are
always "issues down the road" whether or not we see them and that
shouldn't keep us from trying to evolve:)

FileTree was invented (props to Otto Behrens) to smoothly integrate the
use of disk-based repositories into a Monticello development environment
dominated by Monticello tools.

Thierry your work building on the original FileTree has gone a long way
towards making git a viable development option for Pharo.

I think that you'd agree that the next step of evolution for "git-based"
development involves improved tool support and I would think that we
should turn our focus in that direction...

Dale
Ben Coman
2016-02-04 17:39:15 UTC
Permalink
On Fri, Feb 5, 2016 at 1:08 AM, Dale Henrichs
Post by Dale Henrichs
Post by Thierry Goubier
Post by Dale Henrichs
Post by Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the
story is
how Monticello remains compatible and interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello
metadata
with
that from git. How do I, as a Squeak user with Monticello,
ever
get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The
Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you
say
you're
keeping the Monticello metadata. I'm hopelessly confused. How
does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is implemented.
And I agree this may well be the way to go. But ...
Post by Dale Henrichs
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history is
that it has value when you can access the versions listed in it. If you mix
repositories like that, unless you maintain a link to the previous mcz
repository, pre-git versions can't be accessed. So, most of the time, what
we do with a project moved under git is to clone the previous repository,
not take just the current head.
(you'll notice, by the way, that vcs usually work that way when moving
from, say, CVS to git - you move the entire repository, not just the latest
version).
1- should we invest into making that integration of past history a selling
point (but I foresee issues down the road; I've only described one so far,
and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there is
already some code floating around.
Thierry,
My take on this is that we are not trying to "fix" Monticello.
The developers who prefer to use Monticello should continue to use
Monticello and the existing Monticello tools should continue to be
available. Any new formats should provide a migration path from Monticello
and any new tools like a `project list` tool should accommodate projects
based on both git and Monticello repositories.
This particular branch of the conversation has been aimed at trying to
resolve the tension that has been created by including Monticello metadata
in FileTree repositories and I think that "option 3": including past history
in the FileTree repository, but not updating the version history for each
git commit is a workable compromise --- there are always "issues down the
road" whether or not we see them and that shouldn't keep us from trying to
evolve:)
FileTree was invented (props to Otto Behrens) to smoothly integrate the use
of disk-based repositories into a Monticello development environment
dominated by Monticello tools.
Thierry your work building on the original FileTree has gone a long way
towards making git a viable development option for Pharo.
I think that you'd agree that the next step of evolution for "git-based"
development involves improved tool support and I would think that we should
turn our focus in that direction...
Dale
Pertinent is Joel's description of how Microsoft Excel's tipping point
[1] in competing against Lotus123 was when it became able to export to
Lotu123 format. Making it easy for people to move from git back to
Monticello lower the barrier for entry for people to use git.

[1] Scroll down to the second "tipping point"
http://www.joelonsoftware.com/articles/fog0000000052.html

cheers -ben
Esteban A. Maringolo
2016-02-04 17:48:12 UTC
Permalink
Post by Ben Coman
Pertinent is Joel's description of how Microsoft Excel's tipping point
[1] in competing against Lotus123 was when it became able to export to
Lotu123 format. Making it easy for people to move from git back to
Monticello lower the barrier for entry for people to use git.
I always remember this article, but regarding Pharo and Git (or any
other file based SCM) I always thought about it the other way, it
is... the tipping point could be a seamless/smooth adoption of file
based SCMs, with git being the most prominent case.

Esteban A. Maringolo
Dale Henrichs
2016-02-04 18:13:09 UTC
Permalink
Post by Esteban A. Maringolo
Post by Ben Coman
Pertinent is Joel's description of how Microsoft Excel's tipping point
[1] in competing against Lotus123 was when it became able to export to
Lotu123 format. Making it easy for people to move from git back to
Monticello lower the barrier for entry for people to use git.
I always remember this article, but regarding Pharo and Git (or any
other file based SCM) I always thought about it the other way, it
is... the tipping point could be a seamless/smooth adoption of file
based SCMs, with git being the most prominent case.
Esteban A. Maringolo
... and both are important:)
Dale Henrichs
2016-02-04 18:12:14 UTC
Permalink
Post by Ben Coman
On Fri, Feb 5, 2016 at 1:08 AM, Dale Henrichs
Post by Dale Henrichs
Post by Thierry Goubier
Post by Dale Henrichs
Post by Thierry Goubier
On Wed, Feb 3, 2016 at 12:54 AM, Thierry Goubier
Hi Eliot,
....
No it's /not/ the end of the story. The essential part of the
story is
how Monticello remains compatible and interoperable between
dialects. I
haven't seen you account for how you maintain that
compatibility. As
far as I can tell, you propose replacing the Monticello
metadata
with
that from git. How do I, as a Squeak user with Monticello,
ever
get to
look at your package again? As I understand it, moving the metadata
from Monticello commit time to git means that the metadata is in a
format that git determines, not Monticello.
Yes. See below why.
So I don't understand how on the one hand you can say "The
Monticello
metadata in a git repository is redundant and leads to unnecessary
commit conflicts -- end of story ....", which implies you want to
eliminate the Monticello metadata, and on the other hand you
say
you're
keeping the Monticello metadata. I'm hopelessly confused. How
does the
Monticello metadata get reconstituted if it's been thrown away?
What happens to the metadata in the following workflow?
load package P from Monticello repository R into an image
change P, commit via git to local git repository G
load P from G into an image
store P to R via Monticello
It's not a scenario I've specifically worked on, but all the tech is
implemented / implementable to do that perfectly.
The only thing that is problematic there is that the only safe
history is the one generated from git... there are so many MC
packages with broken history that, on mcz packages, you have to
admit that it's not safe to base things on their history.
I'm sorry but I don't accept that. In the Squeak trunk we have history
in our mczs that is correct. Certainly in VMMaker.oscog I have history
that goes back a long time. If bugs have broken history then efforts
should be made to repair that history. But you can't just write off
Monticello history like that.
I don't. You presuppose.
I write tools that work with Monticello repositories, not just yours.
I have to do with what is given to me. On a general level, as a mcz
user, I'll just have to consider that you are as susceptible to be
trusted as with any other mcz producer. This means not much...
... And this is the reason why I am inclined to favor option 3, which
records the package version history as it existed at the point it was
copied into a git repo. When copied back out from the git universe,
create a version history that starts with the original version history
and generates a history of the package in git ....
Which is not very difficult to do given how GitFileTree is implemented.
And I agree this may well be the way to go. But ...
Post by Dale Henrichs
Correct or not, the Monticello version history should be preserved ....
I wonder about that. The property of the Monticello version history is
that it has value when you can access the versions listed in it. If you mix
repositories like that, unless you maintain a link to the previous mcz
repository, pre-git versions can't be accessed. So, most of the time, what
we do with a project moved under git is to clone the previous repository,
not take just the current head.
(you'll notice, by the way, that vcs usually work that way when moving
from, say, CVS to git - you move the entire repository, not just the latest
version).
1- should we invest into making that integration of past history a selling
point (but I foresee issues down the road; I've only described one so far,
and I have seen others)
or
2- into making a better "clone" of a package history, timestamps and
everything when moving a complete repository to git?
Honestly, I'd consider 1- to be the easiest to implement. 2- there is
already some code floating around.
Thierry,
My take on this is that we are not trying to "fix" Monticello.
The developers who prefer to use Monticello should continue to use
Monticello and the existing Monticello tools should continue to be
available. Any new formats should provide a migration path from Monticello
and any new tools like a `project list` tool should accommodate projects
based on both git and Monticello repositories.
This particular branch of the conversation has been aimed at trying to
resolve the tension that has been created by including Monticello metadata
in FileTree repositories and I think that "option 3": including past history
in the FileTree repository, but not updating the version history for each
git commit is a workable compromise --- there are always "issues down the
road" whether or not we see them and that shouldn't keep us from trying to
evolve:)
FileTree was invented (props to Otto Behrens) to smoothly integrate the use
of disk-based repositories into a Monticello development environment
dominated by Monticello tools.
Thierry your work building on the original FileTree has gone a long way
towards making git a viable development option for Pharo.
I think that you'd agree that the next step of evolution for "git-based"
development involves improved tool support and I would think that we should
turn our focus in that direction...
Dale
Pertinent is Joel's description of how Microsoft Excel's tipping point
[1] in competing against Lotus123 was when it became able to export to
Lotu123 format. Making it easy for people to move from git back to
Monticello lower the barrier for entry for people to use git.
[1] Scroll down to the second "tipping point"
http://www.joelonsoftware.com/articles/fog0000000052.html
That was the original rationale for including the Monticello metadata.

Unfortunately the implementation I chose interferes with the natural use
of git, so an alternate solution is needed. I think the technology is
present to be able to implement "option 3" in relatively short order and
we can look forward to this in a future release of FileTree[1].

But we also need to focus on improving the "git" support by the tools so
that developers can leverage some of the advantages that "git" brings to
the tble... developers need to be able to see version history at the
package, class, and method level; they need to be able perform the
"standard" git operations (commit, checkout, branch, diff, push, pull
...) without leaving the image; and they need to be able to do git
merges in the image ....

At least that's my opinion:)

Dale

[1] https://github.com/dalehenrich/filetree/issues/177
Bernhard Pieber
2016-01-30 10:00:50 UTC
Permalink
Hi David,

Just for your information, Cuis uses a file-per-package format. It looks really nice on GitHub and on SourceTree. As far as I can tell, there has not been much merging going, though. So it is entirely possible that it turns out not to work as well as the filetree format when merging.

Cheers,
Bernhard
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is starting to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.
I think more package-based user interfaces would indeed be a very good idea, for browsing and for source code management.
Stef, I have the impression you think that git is popular because it is a new shiny toy. I disagree with this idea. Git is a typical worse-is-better tool. It's good enough for most people, but it still has many shortcomings. It is popular in spite of its shortcomings. It became popular as destination for projects shifting from CVS and Subversion. So it is unlikely to be displaced by a newer, incrementally shinier tools. Anything that will displace it will have to provide an improvement of a similar magnitude as the jump between centralised and distributed version control.
Still, I think it's a good idea not to restrict high level models to what git provides if that's a less than ideal fit to the image model.
I have a lot of ideas to improve browsing and source code management in Pharo. I can make no promises, but I would like to produce something there.
Dale Henrichs
2016-01-30 19:53:55 UTC
Permalink
Post by David Allouche
Thanks Dale for all the explanations.
How Monticello and version control relate in the big picture is starting to make sense for me.
Now, I better understand why filetree ended up uses a file-per-method format, even though that is relatively hostile to git user interfaces optimised for other languages. There is really a need for a file-per-class exchange format, because that would works a lot better with the existing VCS ecosystem.
I agree 100%, but as I've mentioned in another post, I think that
switching to a file-per-class exchange format needs to be a
cross-dialect effort. A format that accommodates the diversity of
Smalltalk class definition and even method metadata while still being
readable is a bit of a challenge - solvable, but challenging....

Personally, I have other fish to fry, but I would be willing to
participate in such an effort if someone else is willing to lead the
charge:)

Dale
Bernhard Pieber
2016-01-30 09:54:23 UTC
Permalink
Dale,

Thanks for your thorough answer. I really appreciate how you include links to helpful articles.

I find the description of the workflow you actually use very enlightening. However, one thing still remains unclear. In the last step, when merging the pull request. How is the unchanged metadata reconciled with the code changes? I just realized that I just don’t know what information is in the Monticello metadata, which is not in the code?

Cheers,
Bernhard
Post by Dale Henrichs
Post by Christophe Demarey
Hi Dale,
I am trying to understand this a little better. If a package containing metadata would be changed using a dialect which cannot interpret the metadata, wouldn’t or at least couldn’t it be broken or lost afterwards for a dialect which tries to interpret the metadata? At least if my assumption is correct that the metadata is related to the code in some way?
Bernhard,
Very good question.
You are absolutely correct, the act of writing a new version of a package using a different dialect would indeed destroy the platform-specific meta data.
However, it is NOT expected that two different dialects (with different disk formats) share the same commit.
- all of the class and instance method source is readable and shared
- the method category is readable and shared
- the name, superclass, instance variables, class instance variables,
pools, class category, and class vars for classes is readable and shared
This is an awful lot common data .... things like traits or namespaces are dialect specific and are ignored by dialects that don't have them when reading the package ...
- use the current branch
- use a new branch
I think that using a new branch is the cleanest option. When I port a project like Zinc[1] to GemStone[2], the master branch is preserved for the Pharo-specific code. I create a gs_master branch where the GemStone code goes. When Sven commits new code in his repo, I merge the changes from the his new master branch into my gs_master and resolve conflicts if any, run tests and I'm done.
The vast mjority of the code base is shared between GemStone and Pharo and only the places where I made changes in porting Zinc to GemStone have the potential for conflicts. Running the tests should highlight any impacts not covered by direct conflicts and if there were some Pharo-specific meta data that I deleted on my branch that causes a test to fail then that is up to me to research and fix.
When I have changes to feed back to Sven, I make those changes on a separate "topic branch"[3] that is merged into gs_master for my use. I then cherry-pick[4] the topic branch commit into a separate topic branch off of the master branch (this picks up only the changes I made on the topic branch) and open a pull request[5] against Svens repository ... if Sven were using travis-ci, tests would be run automatically against a merge of my changes into his master branch ... if the tests are green and the code passes Sven's scrutiny in a code review he merges my proposed changes into his master branch...
GemStone and Pharo share a common disk format (not 100% common), but since git only merges the deltas for a commit, it is relatively easy to keep the meta data differences isolated while still sharing the vast bulk of the code and classes ....
Dale
[1] https://github.com/svenvc/zinc
[2] https://github.com/GsDevKit/zinc
[3] https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows
[4] http://think-like-a-git.net/sections/rebase-from-the-ground-up/cherry-picking-explained.html
[5] https://help.github.com/articles/using-pull-requests/
Dale Henrichs
2016-01-30 19:38:34 UTC
Permalink
Post by Bernhard Pieber
Dale,
Thanks for your thorough answer. I really appreciate how you include links to helpful articles.
I find the description of the workflow you actually use very enlightening. However, one thing still remains unclear. In the last step, when merging the pull request. How is the unchanged metadata reconciled with the code changes? I just realized that I just don’t know what information is in the Monticello metadata, which is not in the code?
Monticello metadata is basically the entire Monticello version history
of the package, it includes direct ancestors, commit comments, the GUID,
etc. For a FileTree repo, the meta data is stashed a separate file ...
The form of the data is actually serialized Smalltalk object graph - a
deeply nested set of Arrays - all written on a single line, so git has
very little chance of being able to merge two files ... The other bit of
Monticello metadata that is maintained by FileTree involves the per
method timestamp and developer initials - these are stored in a method
properties file associated with each class.

In fact is nearly impossible for a human to properly merge to Monticello
version histories, which is why Thierry Goubier's
GitFileTree-MergeDriver[1] is especially useful. Thierry's MergeDriver
fires up a Pharo image (called by git) and performs object surgery to
merge the conflicting version histories and method properties.

Anyway, in the scenario that I described, I simplified the process a bit
and left out some of the gorier details.

When I cherry-picked the commit for my topic branch and merged that
commit into Sven's latest master branch, the Monticello metadata was
certain to conflict, but when you perform a merge on a local git
repository and you've registered Thierry's merge driver with git, the
Monticello metadata is automatically merged.

I also made the assumption that Sven would not have made any additional
commits to his master branch between the time I merged his master branch
and the time I submitted the pull request - the latest version of a file
wins in a merge as long as they share the same immediate previous ancestor.

If Sven had made a new commit, the Monticello metadata would have
conflicted ...

For the record, these "guaranteed conflicts" are the only reason that
"removing Monticello metadata" enters into the conversation .... git
maintains it's own version history and method properties, so for
developers who are "exclusively using git" the Monticello metadata is an
annoying source of "meaningless conflicts." All of the information that
is present in Monticello meta data is present in git in another form

As I said, Thierry Goubier's magical GitFileTree-MergeDriver[1]
eliminates the conflict pain when the merges are performed in a local
git repository, but on GitHub, merging the Monticello metadata is a
constant thorn in one's side, since it becomes necessary to perform a
manual merge (and leverage Thierry's Merge Driver) when Monticello
metadata is involved.

Dale

[1] https://github.com/ThierryGoubier/GitFileTree-MergeDriver
Ben Coman
2016-01-31 04:27:02 UTC
Permalink
On Sun, Jan 31, 2016 at 3:38 AM, Dale Henrichs
Post by Bernhard Pieber
Dale,
Thanks for your thorough answer. I really appreciate how you include links
to helpful articles.
I find the description of the workflow you actually use very enlightening.
However, one thing still remains unclear. In the last step, when merging the
pull request. How is the unchanged metadata reconciled with the code
changes? I just realized that I just don’t know what information is in the
Monticello metadata, which is not in the code?
Monticello metadata is basically the entire Monticello version history of
the package, it includes direct ancestors, commit comments, the GUID, etc.
For a FileTree repo, the meta data is stashed a separate file ... The form
of the data is actually serialized Smalltalk object graph - a deeply nested
set of Arrays - all written on a single line, so git has very little chance
of being able to merge two files
What would it take to have the meta data spread out over multiple
lines (if that would work better with git?)

cheers -ben
Dale Henrichs
2016-02-02 19:38:58 UTC
Permalink
Post by Ben Coman
On Sun, Jan 31, 2016 at 3:38 AM, Dale Henrichs
Post by Bernhard Pieber
Dale,
Thanks for your thorough answer. I really appreciate how you include links
to helpful articles.
I find the description of the workflow you actually use very enlightening.
However, one thing still remains unclear. In the last step, when merging the
pull request. How is the unchanged metadata reconciled with the code
changes? I just realized that I just don’t know what information is in the
Monticello metadata, which is not in the code?
Monticello metadata is basically the entire Monticello version history of
the package, it includes direct ancestors, commit comments, the GUID, etc.
For a FileTree repo, the meta data is stashed a separate file ... The form
of the data is actually serialized Smalltalk object graph - a deeply nested
set of Arrays - all written on a single line, so git has very little chance
of being able to merge two files
What would it take to have the meta data spread out over multiple
lines (if that would work better with git?)
Hmmm I suppose that if the meta data were represented in STON (using
pretty print) the changes might be more mergable, but I think that
Thierry's algorithm might be still to a proper merge, since it takes
"object surgery" to get things right .... STON might make it possible
for a human to do the necessary edits though ...

Dale
Eliot Miranda
2016-02-02 20:56:01 UTC
Permalink
On Tue, Feb 2, 2016 at 11:38 AM, Dale Henrichs <
Post by Dale Henrichs
Post by Ben Coman
On Sun, Jan 31, 2016 at 3:38 AM, Dale Henrichs
Post by Bernhard Pieber
Dale,
Thanks for your thorough answer. I really appreciate how you include links
to helpful articles.
I find the description of the workflow you actually use very enlightening.
However, one thing still remains unclear. In the last step, when merging the
pull request. How is the unchanged metadata reconciled with the code
changes? I just realized that I just don’t know what information is in
the
Monticello metadata, which is not in the code?
Monticello metadata is basically the entire Monticello version history
of
the package, it includes direct ancestors, commit comments, the GUID, etc.
For a FileTree repo, the meta data is stashed a separate file ... The form
of the data is actually serialized Smalltalk object graph - a deeply nested
set of Arrays - all written on a single line, so git has very little chance
of being able to merge two files
What would it take to have the meta data spread out over multiple
lines (if that would work better with git?)
Hmmm I suppose that if the meta data were represented in STON (using
pretty print) the changes might be more mergable, but I think that
Thierry's algorithm might be still to a proper merge, since it takes
"object surgery" to get things right .... STON might make it possible for a
human to do the necessary edits though ...
Please tell me this isn't about line endings? Why can't the version
history be written with lf line endings? That's hardly a bone of
contention is it?

_,,,^..^,,,_
best, Eliot
Thierry Goubier
2016-02-02 21:27:48 UTC
Permalink
Post by Eliot Miranda
On Tue, Feb 2, 2016 at 11:38 AM, Dale Henrichs
On Sun, Jan 31, 2016 at 3:38 AM, Dale Henrichs
Dale,
Thanks for your thorough answer. I really appreciate how
you include links
to helpful articles.
I find the description of the workflow you actually use
very enlightening.
However, one thing still remains unclear. In the last
step, when merging the
pull request. How is the unchanged metadata reconciled
with the code
changes? I just realized that I just don’t know what
information is in the
Monticello metadata, which is not in the code?
Monticello metadata is basically the entire Monticello
version history of
the package, it includes direct ancestors, commit comments,
the GUID, etc.
For a FileTree repo, the meta data is stashed a separate
file ... The form
of the data is actually serialized Smalltalk object graph -
a deeply nested
set of Arrays - all written on a single line, so git has
very little chance
of being able to merge two files
What would it take to have the meta data spread out over multiple
lines (if that would work better with git?)
Hmmm I suppose that if the meta data were represented in STON (using
pretty print) the changes might be more mergable, but I think that
Thierry's algorithm might be still to a proper merge, since it takes
"object surgery" to get things right .... STON might make it
possible for a human to do the necessary edits though ...
Please tell me this isn't about line endings? Why can't the version
history be written with lf line endings? That's hardly a bone of
contention is it?
No, it's a lot more complex than that. It's simply hard to do a
merge-able text log format. Git also makes a mess of json / ston files,
simply because they are structured data in what appears to be text
files, git treat them as text files, and three-way line by line merging
does not work.

The only hope you can have is to make a format which is easier for a
human to correct when git creates a conflict when merging ... in short,
something which would only make it easier as long as the history is
limited to 50 versions and a single developper :(

Now, given how much cleaner, safer and sound is extracting that data
from the dvcs log, I'll never bother with that sort of things again.

Thierry
Dale Henrichs
2016-01-28 19:17:28 UTC
Permalink
Post by Sven Van Caekenberghe
Post by Christophe Demarey
I read too fast.
But my comment is still true.
We also want a new version browser. Monticello browser is too tied to ... Monticello. We would like a better approach for both git and monticello.
It is totally wrong to put Git and Monticello at the same level, they are related but different things.
Monticello models our code and your changes to it. It also has a way of putting a package into a persistent representation that can be copied around. An MC server is nothing more than storage.
Filetree is another way of doing that, using a different representation, but it still needs MC for the high level stuff. Git is versioned storage. And there is a conflict there, yes.
But throwing away the MC code and changes model would leave you with file outs. You do not want to go there, really. Never, ever.
Sven,

If you look at my presentation on "Practical Git for Smalltalk"[1] from
2012, I take pains to differentiate between the Monticello package model
(definitions and snapshots) and the Monticello repository model (mcz files).

Noone is proposing to throw away the Monticello package model (to my
knowledge).

So in these discussions about supporting git we are talking about a
model where Filetree is still being used for Monticello packages on top
of a git storage strata ...

I have brought up the point about a "Metacello Project Browser" because
both git-based and mcz-based projects can benefit from a higher level of
management:

- commit all dirty packages, then update and commit the
ConfigurationOf for mcz-based projects, OR
- save all dirty packages and perform a git commit for git-based projects

- perform a diff on all dirty packages in a project ...

- etc.

Dale


[1]
http://www.stic.st/wp-content/conferences/2012/Wednesday/1415-Practical_Git_for_Smalltalk-Henrichs.pdf
Thierry Goubier
2016-01-27 08:03:48 UTC
Permalink
Hi Dale,

then a question: how does one query a baselineOf or a configOf to retrieve
the current project state (i.e. packages) loaded in the image?

Thierry
Post by Dale Henrichs
I've been wondering no-one in the Pharo community has started building a
"Project Browser" to replace the "Package Browser" .... I've had a "project
browser" in tODE for several years now and you work with a manageble list
[image: project list]
instead of a giant list of packages like the following (only half of the
[image: package list]
Like a package browser, the project shows up as dirty whenever a package
in the project is dirtied and all of the operations `save`, `diff`, etc.
apply equally well to projects instead of packages .... with the added
bonus of performing the action on the the project itself instead of having
to view diffs on each dirty package or save each dirty package ... for the
git-based repos, a single git commit is performed after all of the dirty
packages are committed , thus allowing git to track the moves of methods
between packages and combining all of the changes for a unit of work in a
single commit ...
Also, in the `project list`, the loaded version and blessing (1.0.5.4
[release]) or SHA of the git commit and branch name (b674ad0 [master]) is
Gofer 1.0.5.4 [release]
Grease b674ad0 [master]
... and this is very useful information. Metacello provides an api for
this information and a project-based workflow is just much simpler to work
in than a package-based workflow...
I'm willing to share code from tODE and Metacello if anyone is interested
... and I think that a tool like this is really required to properly
support a git (or any other disk-based SCM) workflow ...
Perhaps work is already underway, but whenever I see folks talking about
the funky Monticello Browser interface, I can't help thinking about the
fact that I haven't used an interface like that for several years now and
it's time for the tooling to move forward a bit more:)
Dale
checkbox for displaying non-dirty packages - so we only display the
short list in the usual case. (??)
Ah! That would be very helpful. Particularly annoying is when you commit one
of your project's dirty packages, the list auto-scrolls to the package's new
location among the non-dirty packages, and you have to manually scroll back
to the top of the package list to find your next dirty package.
-----
Cheers,
Sean
--
View this message in context: http://forum.world.st/Contributing-to-Pharo-tp4871604p4873422.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
Dale Henrichs
2016-01-27 14:41:53 UTC
Permalink
Post by Christophe Demarey
Hi Dale,
then a question: how does one query a baselineOf or a configOf to
retrieve the current project state (i.e. packages) loaded in the image?
Thierry
Thierry,

Good question:)

To start with in tODE I have a class
TDMetacelloProjectRegistrationDefinition which is part of the following
hierarchy:

Object
TDDefinition
TDPackageBasedDefinition
TDAbstractProjectRegistrationDefinition
TDGitProjectRegistrationDefinition
TDMetacelloProjectRegistrationDefinition
TDMetacelloRegistrationDefinition
TDMetacelloBaselineRegistrationDefinition
TDMetacelloConfigurationRegistrationDefinition
TDMetacelloHybridRegistrationDefinition
TDMetacelloUnloadedProjectRegistrationDefinition

The method TDMetacelloProjectRegistrationDefinition>>workingCopies[1]
calculates the list of MCWorkingCopy instances for the loaded packages
associated with a project (including the configurationof or
baselineof... or both).

The key Metacello methods called include:

MetacelloProjectRegistration>>projectVersion[2]
MetacelloMCVersion>>packages[3]
MetacelloPackageSpec>>workingCopy[4]
MetacelloMCProjectSpec>>projectPackage[5]

This is really the key information for doing all of the other operations
for a "Project Browser" like "save all dirty packages and commit" and
"give me diffs for all dirty packages in project"...

Dale

[1]
https://github.com/dalehenrich/tode/blob/master/repository/Topez-Server-Core.package/TDMetacelloProjectRegistrationDefinition.class/instance/workingCopies.st
[2]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-Core.package/MetacelloProjectRegistration.class/instance/projectVersion.st
[3]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloMCVersion.class/instance/packages.st
[4]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloPackageSpec.class/instance/workingCopy.st
[5]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloMCProjectSpec.class/instance/projectPackage.st
Dale Henrichs
2016-01-27 16:16:33 UTC
Permalink
Thierry,

I should mention that to get a registration for a project in the first
place, you call:

MetacelloProjectRegistration
class>>registrationForClassNamed:ifAbsent:[1]

... and yes you pass in the full class name for the ConfigurationOf or
BaselineOf ...

Dale

[1]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-Core.package/MetacelloProjectRegistration.class/class/registrationForClassNamed.ifAbsent..st
Post by Dale Henrichs
Post by Christophe Demarey
Hi Dale,
then a question: how does one query a baselineOf or a configOf to
retrieve the current project state (i.e. packages) loaded in the image?
Thierry
Thierry,
Good question:)
To start with in tODE I have a class
TDMetacelloProjectRegistrationDefinition which is part of the
Object
TDDefinition
TDPackageBasedDefinition
TDAbstractProjectRegistrationDefinition
TDGitProjectRegistrationDefinition
TDMetacelloProjectRegistrationDefinition
TDMetacelloRegistrationDefinition
TDMetacelloBaselineRegistrationDefinition
TDMetacelloConfigurationRegistrationDefinition
TDMetacelloHybridRegistrationDefinition
TDMetacelloUnloadedProjectRegistrationDefinition
The method TDMetacelloProjectRegistrationDefinition>>workingCopies[1]
calculates the list of MCWorkingCopy instances for the loaded packages
associated with a project (including the configurationof or
baselineof... or both).
MetacelloProjectRegistration>>projectVersion[2]
MetacelloMCVersion>>packages[3]
MetacelloPackageSpec>>workingCopy[4]
MetacelloMCProjectSpec>>projectPackage[5]
This is really the key information for doing all of the other
operations for a "Project Browser" like "save all dirty packages and
commit" and "give me diffs for all dirty packages in project"...
Dale
[1]
https://github.com/dalehenrich/tode/blob/master/repository/Topez-Server-Core.package/TDMetacelloProjectRegistrationDefinition.class/instance/workingCopies.st
[2]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-Core.package/MetacelloProjectRegistration.class/instance/projectVersion.st
[3]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloMCVersion.class/instance/packages.st
[4]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloPackageSpec.class/instance/workingCopy.st
[5]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloMCProjectSpec.class/instance/projectPackage.st
Thierry Goubier
2016-01-28 08:50:51 UTC
Permalink
Hi Dale,
Post by Dale Henrichs
Post by Christophe Demarey
Hi Dale,
then a question: how does one query a baselineOf or a configOf to
retrieve the current project state (i.e. packages) loaded in the image?
Thierry
Thierry,
Good question:)
To start with in tODE I have a class
TDMetacelloProjectRegistrationDefinition which is part of the following
Object
TDDefinition
TDPackageBasedDefinition
TDAbstractProjectRegistrationDefinition
TDGitProjectRegistrationDefinition
TDMetacelloProjectRegistrationDefinition
TDMetacelloRegistrationDefinition
TDMetacelloBaselineRegistrationDefinition
TDMetacelloConfigurationRegistrationDefinition
TDMetacelloHybridRegistrationDefinition
TDMetacelloUnloadedProjectRegistrationDefinition
The method TDMetacelloProjectRegistrationDefinition>>workingCopies[1]
calculates the list of MCWorkingCopy instances for the loaded packages
associated with a project (including the configurationof or baselineof...
or both).
MetacelloProjectRegistration>>projectVersion[2]
MetacelloMCVersion>>packages[3]
MetacelloPackageSpec>>workingCopy[4]
MetacelloMCProjectSpec>>projectPackage[5]
This is really the key information for doing all of the other operations
for a "Project Browser" like "save all dirty packages and commit" and "give
me diffs for all dirty packages in project"...
I was just asking that because I'd like to see that information merged as
much as possible; separating concepts just strikes me as unneeded
complexity, at least for baselines. Configurations are a bit different,
since a configuration can snapshot multiple versions. But I'd really like
to make it simple and obvious how to setup a project.

I'll make a few tries, and see what I can come up with. When I'll have the
time :(

Thierry
Post by Dale Henrichs
Dale
[1]
https://github.com/dalehenrich/tode/blob/master/repository/Topez-Server-Core.package/TDMetacelloProjectRegistrationDefinition.class/instance/workingCopies.st
[2]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-Core.package/MetacelloProjectRegistration.class/instance/projectVersion.st
[3]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloMCVersion.class/instance/packages.st
[4]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloPackageSpec.class/instance/workingCopy.st
[5]
https://github.com/dalehenrich/metacello-work/blob/master/repository/Metacello-MC.package/MetacelloMCProjectSpec.class/instance/projectPackage.st
Dale Henrichs
2016-01-28 19:55:06 UTC
Permalink
Post by Christophe Demarey
Hi Dale,
2016-01-27 15:41 GMT+01:00 Dale Henrichs
Hi Dale,
then a question: how does one query a baselineOf or a configOf
to retrieve the current project state (i.e. packages) loaded
in the image?
Thierry
Thierry,
Good question:)
To start with in tODE I have a class
TDMetacelloProjectRegistrationDefinition which is part of the
Object
TDDefinition
TDPackageBasedDefinition
TDAbstractProjectRegistrationDefinition
TDGitProjectRegistrationDefinition
TDMetacelloProjectRegistrationDefinition
TDMetacelloRegistrationDefinition
TDMetacelloBaselineRegistrationDefinition
TDMetacelloConfigurationRegistrationDefinition
TDMetacelloHybridRegistrationDefinition
TDMetacelloUnloadedProjectRegistrationDefinition
The method
TDMetacelloProjectRegistrationDefinition>>workingCopies[1]
calculates the list of MCWorkingCopy instances for the loaded
packages associated with a project (including the configurationof
or baselineof... or both).
MetacelloProjectRegistration>>projectVersion[2]
MetacelloMCVersion>>packages[3]
MetacelloPackageSpec>>workingCopy[4]
MetacelloMCProjectSpec>>projectPackage[5]
This is really the key information for doing all of the other
operations for a "Project Browser" like "save all dirty packages
and commit" and "give me diffs for all dirty packages in project"...
I was just asking that because I'd like to see that information merged
as much as possible; separating concepts just strikes me as unneeded
complexity, at least for baselines.
Not sure what you mean by "separating concepts and unneeded complexity".
I provided a picture of some of the abstractions that I am using in tODE
... these abstractions are used in a number of commands, but I also
tried to provide you with the basic implementation used in tODE in case
you were interested in the fact that tODE includes the BaselineOf
package in the list of packages for a project...

The simple answer is MetacelloMCVersion>>packages[3]
Post by Christophe Demarey
Configurations are a bit different, since a configuration can snapshot
multiple versions. But I'd really like to make it simple and obvious
how to setup a project.
I'm not sure what you are talking about here.

In Metacello a BaselineOf has a single version and the
MetacelloMCVersion class manages the version ... Yes, a Configuration
has multiple versions and each of those versions is an instance of
MetacelloMCVersion....

Perhaps when you have time, you can provide a bit more information about
what you are trying to do ...

I apologize if I provided too much information but when working with
folks in different time zones, I am in the habit of packing as much
information in a message as possible, because it's often a 24 hour
turnaround for responses... (as it is in this case:)

Dale
Thierry Goubier
2016-01-28 20:16:50 UTC
Permalink
Post by Dale Henrichs
Post by Christophe Demarey
Hi Dale,
2016-01-27 15:41 GMT+01:00 Dale Henrichs
Hi Dale,
then a question: how does one query a baselineOf or a configOf
to retrieve the current project state (i.e. packages) loaded
in the image?
Thierry
Thierry,
Good question:)
To start with in tODE I have a class
TDMetacelloProjectRegistrationDefinition which is part of the
Object
TDDefinition
TDPackageBasedDefinition
TDAbstractProjectRegistrationDefinition
TDGitProjectRegistrationDefinition
TDMetacelloProjectRegistrationDefinition
TDMetacelloRegistrationDefinition
TDMetacelloBaselineRegistrationDefinition
TDMetacelloConfigurationRegistrationDefinition
TDMetacelloHybridRegistrationDefinition
TDMetacelloUnloadedProjectRegistrationDefinition
The method
TDMetacelloProjectRegistrationDefinition>>workingCopies[1]
calculates the list of MCWorkingCopy instances for the loaded
packages associated with a project (including the configurationof
or baselineof... or both).
MetacelloProjectRegistration>>projectVersion[2]
MetacelloMCVersion>>packages[3]
MetacelloPackageSpec>>workingCopy[4]
MetacelloMCProjectSpec>>projectPackage[5]
This is really the key information for doing all of the other
operations for a "Project Browser" like "save all dirty packages
and commit" and "give me diffs for all dirty packages in project"...
I was just asking that because I'd like to see that information merged
as much as possible; separating concepts just strikes me as unneeded
complexity, at least for baselines.
Not sure what you mean by "separating concepts and unneeded complexity".
I provided a picture of some of the abstractions that I am using in tODE
... these abstractions are used in a number of commands, but I also
tried to provide you with the basic implementation used in tODE in case
you were interested in the fact that tODE includes the BaselineOf
package in the list of packages for a project...
This is me trying to express something and being unable to make it clear
:) I'll have something at one point later. Or maybe not. Doesn't really
matter; I'll wait until it becomes clearer to me.
Post by Dale Henrichs
The simple answer is MetacelloMCVersion>>packages[3]
Thanks. I'll try something.
Post by Dale Henrichs
Post by Christophe Demarey
Configurations are a bit different, since a configuration can snapshot
multiple versions. But I'd really like to make it simple and obvious
how to setup a project.
I'm not sure what you are talking about here.
In Metacello a BaselineOf has a single version and the
MetacelloMCVersion class manages the version ... Yes, a Configuration
has multiple versions and each of those versions is an instance of
MetacelloMCVersion....
Perhaps when you have time, you can provide a bit more information about
what you are trying to do ...
I was just looking at it from the angle of having to explain it to one
of my students (non-smalltalkers)... The simplest path to creating a
project.

I guess one must have a tool like Versionner.
Post by Dale Henrichs
I apologize if I provided too much information but when working with
folks in different time zones, I am in the habit of packing as much
information in a message as possible, because it's often a 24 hour
turnaround for responses... (as it is in this case:)
Don't worry about that, you already provided a lot, certainly enough for
what I was looking for: a way to organize groups of packages in the
image out of the Metacello baselines and configurations present, instead
of hacking stuff based on catalog keywords and package name prefix.

Now, it's time to implement... If I find that time :(

Thierry
Dale Henrichs
2016-01-28 20:38:11 UTC
Permalink
Post by Thierry Goubier
Post by Dale Henrichs
Post by Christophe Demarey
Configurations are a bit different, since a configuration can snapshot
multiple versions. But I'd really like to make it simple and obvious
how to setup a project.
I'm not sure what you are talking about here.
In Metacello a BaselineOf has a single version and the
MetacelloMCVersion class manages the version ... Yes, a Configuration
has multiple versions and each of those versions is an instance of
MetacelloMCVersion....
Perhaps when you have time, you can provide a bit more information about
what you are trying to do ...
I was just looking at it from the angle of having to explain it to one
of my students (non-smalltalkers)... The simplest path to creating a
project.
I guess one must have a tool like Versionner.
From this perspective you should look at the MetacelloToolBox and
specifically MetacelloToolBox
class>>createBaselineOf:requiredProjects:packages:repositories:dependencies:groups:.


Versionner uses the MetacelloToolBox api and about 3 months ago I added
the basic support for creating a BaselineOf as I added a `project new`
command to tODE ... I would be glad to expand the MetacelloToolBox api
for BaselineOf if folks ask for specific functionality ...

The MetacelloToolBox is meant to be the common point for
programmatically creating and updating ConfigurationOf and BaselineOf ...

Dale
Stephan Eggermont
2016-01-16 13:57:45 UTC
Permalink
Post by David Allouche
That does not make sense to me. I guess it will, when I understand Monticello better.
Ideally, you start with the fresh, latest clean Pharo image to start
working on an issue and make the changes in one go to create a slice.
Then you only have to select the dirty packages.

If the changes you are making risk killing the image, or there are
simply a lot of them, you might commit intermediate steps where you
detail your thought process/assumptions. The slice for the issue then
can consist of packages that are already committed.

Stephan
Continue reading on narkive:
Loading...