Patch for problems with projects that are moved around a lot...
Dirk
vss2svn at nogga.de
Thu Feb 1 16:18:53 CST 2007
Hi Stephen,
thanks for looking this up. This is indeed a problem. MOVES and esp.
ARCHIVE / RESTORES are kind of lacking support up to now, since it was
not used so much in the main contributors context.
The GetChildRecs function is used in two contexts: MergeParentData and
MergeMoveData. And in both context a match of more than one file is an
error. Previoulsy, we had this time-constraint that led to numerous
orphaned files, because of missing matching rows. Since we use the
"widetime" branch, things are better in this area.
As you write yourself, your fix will still give a matching pair, even if
the two actions are years away. What would happen if we had picked up
the wrong action in the first place?
Two ideas come into my mind in order to solve the problem:
1.) small timeframe for move actions:
the need for the big timeframe is due to the fact, that commiting files
into the archive and finally updating the parent is not a distinct
action. If you have folders with very large files, it can take some time
to transfer via the network, create the physical file and so on. The
parent is updated after the last file is committed to the folder.
In a move action the move is completed more or less in a very short time
frame, since no files have to be transferred, no new files created and
so on. Only source and destination parents have to be changed. So we can
expect short timeframes here.
2.) In a move situation the database rows must be next to each other.
Again, since the move action is done in one go, even if the timestamp
for both actions are not the same, in the sorted database the two move
actions must be next to each other, since no other vss action could
interfere (hopefully)
I will think about this a little further.
Good to here, that your conversion is now nearly successful.
Best regards
Dirk
Stephen Lee schrieb:
> If a project is moved more than once by the same person, this causes
> problems (on vss2svn trunk).
>
> You see log messages like this:
> TASK: MERGEMOVEDATA
> ERROR -- Multiple chidl recs for parent MOVE rec '36'
> at vss2svn.pl line 620
> ERROR -- Multiple chidl recs for parent MOVE rec '116'
> at vss2svn.pl line 620
> ERROR -- Multiple chidl recs for parent MOVE rec '17614'
> at vss2svn.pl line 620
>
> This seems to be because GetChildRecs will match the "other half" of
> any move done by the same user to the same folder irrespective of how
> far apart the timestamps are. If a project has been moved back and
> forth, it pulls in both (or all) records and "does the wrong thing".
>
> Since both calls to GetChildRecs complain if they get multiple records
> (and thus presumably have no valid reason to do so), I fixed this by
> adding a "LIMIT 1" to the SQL. Together with the 'ORDER BY' this will
> then return only the record with the closest timestamp. It should
> probably have some sensible limit on timestamp though, as in some
> cases it could match events that happened months or even years apart.
>
>
> I also had a project that had been moved into a folder that was later
> deleted, and moved back out again. i.e. equivalent of:
>
> Create $/Project/Subproj
> Move to $/Project/Tmpproj/Subproj
> Move to $/Project/Subproj (by a different user)
> Destroy $/Project/Tmpproj
>
> The parent records from "the other half of the move" were lost with
> Tmpproj. Other projects initially created in Tmpproj were orphaned,
> and I see that there is code in the move to try and "do the right
> thing" by moving to orphaned in such a case.
>
> The change to Dumpfile.pm in move_handler ensures that it will create
> the appropriate subfolder of orphaned (based on the same thing in the
> add_handler)
>
> Finally, when the project is moved back out of its orphaned location,
> it was not getting picked up correctly. Instead an old, inactive copy
> of the project was being shared.
>
> The change to ActionHandler.pm's move_handler changes how it
> determines if it needs to move from orphaned... now based on active
> parents rather than physinfo->{order} (which seems to be all parents
> an item has ever had). At the same time I also promoted the source
> comment
> # Don't know from where to move. Share it there instead
> into an error
> $self->{errmsg} .= "Don't know from where to move $physname.
> Sharing instead.\n";
>
>
> This works well on my database, which now fully matches SourceSafe
> except for one file on a couple of branches (that file had such
> obscure things done to it I have trouble following it by hand!)
>
>
> There are a few possible caveats with the changes, in particular where
> I've tweaked code that was obviously in there to fix problems that did
> not exist in my repository... would be interested to know if it
> improves or has adverse effects on other repositories.
>
> I also attach a batch file that I had used to create a test SourceSafe
> database to trigger this problem (not quite a minimal test case - I'd
> kept on adding things on to try and match the history of what the real
> project had done, until I realised the critical part was the move done
> by a different user - 'Guest' in the batch file).
>
> ------------------------------------------------------------------------
>
> Index: script/vss2svn.pl
> ===================================================================
> --- script/vss2svn.pl (revision 285)
> +++ script/vss2svn.pl (working copy)
> @@ -550,6 +550,7 @@
> AND author = ?
> ORDER BY
> ABS(? - timestamp)
> +LIMIT 1
> EOSQL
>
> my $sth = $gCfg{dbh}->prepare($sql);
> Index: script/Vss2Svn/ActionHandler.pm
> ===================================================================
> --- script/Vss2Svn/ActionHandler.pm (revision 285)
> +++ script/Vss2Svn/ActionHandler.pm (working copy)
> @@ -413,14 +413,17 @@
>
> if (!defined $row->{parentphys}) {
> # Check if this is an orphaned item
> - if (scalar @{$physinfo->{order}} == 1) {
> - $row->{parentphys} = $physinfo->{order}[0];
> + # or if there is otherwise only one active item path
> + my @parents = $self->_get_active_parents ($row->{physname});
> + if (scalar @parents == 1) {
> + $row->{parentphys} = $parents[0];
> } else {
> - # Don't know from where to move. Share it there instead
> + $self->{errmsg} .= "Don't know from where to move $physname. Sharing instead.\n";
> $row->{parentphys} = $row->{info};
> $row->{info} = undef;
> $self->{action} = 'SHARE';
> - return $self->_share_handler();
> + $self->_share_handler();
> + return 0;
> }
> }
>
> Index: script/Vss2Svn/Dumpfile.pm
> ===================================================================
> --- script/Vss2Svn/Dumpfile.pm (revision 285)
> +++ script/Vss2Svn/Dumpfile.pm (working copy)
> @@ -390,7 +390,20 @@
> . "missing recover; skipping");
> return 0;
> }
> -
> +
> + my $success = $self->{repository}->exists_parent ($newpath);
> + if(!defined($success)) {
> + $self->add_error("Attempt to move item '$itempath' to '$newpath' at "
> + . "revision $data->{revision_id}, but path consistency failure at dest");
> + return 0;
> + }
> + elsif ($success == 0) {
> + $self->add_error("Parent path missing while trying to move "
> + . "item '$itempath' to '$newpath' at "
> + . "revision $data->{revision_id}: adding missing parents");
> + $self->_create_svn_path ($nodes, $newpath);
> + }
> +
> my $node = Vss2Svn::Dumpfile::Node->new();
> $node->set_initial_props($newpath, $data);
> $node->{action} = 'add';
>
> ------------------------------------------------------------------------
>
> set SSDIR=D:\sstestdb
> rmdir /s /q %SSDIR%\data
> mkdir %SSDIR%\data
> mkss %SSDIR%\data
> ddconv %SSDIR%\data
> ss cp $/ -YAdmin
> ss create myproject -YAdmin -Crevision1
> pause
> ss create myproject/mysubproj -YAdmin -Crevision1
> pause
> echo revision1 > testfile.txt
> ss cp $/myproject/mysubproj -YAdmin
> ss add testfile.txt -YAdmin -Crevision1 -W
> pause
> ss cp $/ -YAdmin
> ss rename myproject/mysubproj myproject/subproj -YAdmin
> pause
> ss rename myproject project -YAdmin
> pause
> ss move project/subproj subproj -YAdmin
> pause
> ss create project/subproj -YAdmin -Ctemporary
> pause
> ss move subproj project/subproj/subproj -YAdmin
> pause
> ss cp $/ -YAdmin
> ss create project/subproj/subproj2 -YAdmin -Csubproj2
> pause
> ss cp $/project/subproj/subproj2 -YAdmin
> ss share $/project/subproj/subproj/testfile.txt -YAdmin -G-
> pause
> ss cp $/project/subproj/subproj -YAdmin
> ss checkout testfile.txt -YAdmin -G-
> echo revision2 > testfile.txt
> ss checkin testfile.txt -YAdmin -W -Crevision2
> pause
> ss cp $/ -YAdmin
> ss rename project/subproj project/tmpproj -YAdmin
> pause
> ss move project/tmpproj/subproj project/subproj -YGuest
> pause
> ss move project/tmpproj/subproj2 project/subproj2 -YAdmin
> pause
> ss destroy project/tmpproj -YAdmin
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> vss2svn-users mailing list
> Project homepage:
> http://www.pumacode.org/projects/vss2svn/
> Subscribe/Unsubscribe/Admin:
> http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org
> Mailing list web interface (with searchable archives):
> http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user
More information about the vss2svn-users
mailing list