From cyreve at gmail.com Sat Mar 8 07:50:13 2008 From: cyreve at gmail.com (Richard Hughes) Date: Sat Mar 8 07:50:23 2008 Subject: Infinite recursion in mergeparentdata In-Reply-To: <46AA04F7.7080709@nogga.de> References: <1565404365.20070724203840@rhughes.net> <46AA04F7.7080709@nogga.de> Message-ID: <257413283.20080308125013@rhughes.net> This is a resurrection of a thread from 6 months ago, for which I now have more information. The basic background was: >> I'm getting a crash during conversion when GetPathDepth recurses >> itself forever (Perl exhausts its address space then segfaults). It >> appears to be caused by a pair of physicalaction rows which refer to >> each other I've now put some effort into this and have created the following (which are all at the bottom of this e-mail): 1. A minimal testcase, in the form of a batch file which should be run on a brand new, totally empty sourcesafe database and which will create something which crashes vss2svn (r336). 2. The entire contents of the PhysicalAction table resulting from the above database, as it exists upon entry to MergeParentData(). 3. A patch against vss2svn.pl r336 which makes the problem go away. I'm reluctant to use the word 'fixes' because I don't understand enough about what the parentdata column is used for to be at all sure that I've done the right thing. I have, however, confirmed that the resultant svn repository has the correct history for my testcase. I'd be very appreciative if somebody knowledgable could review this patch. It does still throw the warning "ERROR -- Multiple child recs for parent MOVE rec '7' at vss2svn.pl line 740", but I believe this to be harmless because the timestamp check shortly thereafter sorts it all out correctly. Richard. -----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<----- **** make-broken-db.bat setlocal set SSDIR=c:\vss\broken ss create -YAdmin -I- $/first sleep 2 ss create -YAdmin -I- $/first/second sleep 2 ss create -YAdmin -I- $/third sleep 2 ss move -YAdmin $/first/second $/third/second sleep 2 ss move -YAdmin $/third/second $/first/second sleep 2 ss move -YAdmin $/third $/first/second/third endlocal -----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<----- **** vss_data.physicalaction.txt [I changed my mind and added this as an attachment instead so that the line breaks don't get mangled and because it's not important that it be archived.] -----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<----- **** vss2svn.pl.r336.patch Index: vss2svn.pl =================================================================== --- vss2svn.pl (revision 336) +++ vss2svn.pl (working copy) @@ -599,10 +599,11 @@ parentdata > 0 AND physname = ? AND actiontype = ? + AND timestamp <= ? EOSQL my $sth = $gCfg{dbh}->prepare($sql); - $sth->execute( @{ $row }{qw(parentphys actiontype)} ); + $sth->execute( @{ $row }{qw(parentphys actiontype timestamp)} ); $parents = $sth->fetchall_arrayref( {} ); $maxParentDepth = 0; -------------- next part -------------- action_id physname version parentphys actiontype itemname itemtype timestamp author is_binary info priority sortkey parentdata label comment ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- 0 AAAAAAAA 1 ADD / 1 1204934932 Admin 0 1 AAAAAAAA 0 1 BAAAAAAA AAAAAAAA ADD first/ 1 1204934935 Admin 0 1 AAAAAAAB 1 2 DAAAAAAA AAAAAAAA ADD third/ 1 1204934939 Admin 0 1 AAAAAAAD 1 3 DAAAAAAA AAAAAAAA MOVE_TO third/ 1 1204934946 Admin 0 5 AAAAAAAD 1 4 BAAAAAAA 1 ADD first/ 1 1204934935 Admin 0 1 AAAAAAAB 0 5 CAAAAAAA BAAAAAAA ADD second/ 1 1204934937 Admin 0 1 AAAAAAAC 1 6 CAAAAAAA BAAAAAAA MOVE_TO second/ 1 1204934942 Admin 0 5 AAAAAAAC 1 7 CAAAAAAA BAAAAAAA MOVE_FROM second/ 1 1204934944 Admin 0 5 AAAAAAAC 1 8 CAAAAAAA 1 ADD second/ 1 1204934937 Admin 0 1 AAAAAAAC 0 9 DAAAAAAA CAAAAAAA MOVE_FROM third/ 1 1204934946 Admin 0 5 AAAAAAAD 1 10 DAAAAAAA 1 ADD third/ 1 1204934939 Admin 0 1 AAAAAAAD 0 11 CAAAAAAA DAAAAAAA MOVE_FROM second/ 1 1204934942 Admin 0 5 AAAAAAAC 1 12 CAAAAAAA DAAAAAAA MOVE_TO second/ 1 1204934944 Admin 0 5 AAAAAAAC 1 From toby at etjohnson.us Mon Mar 10 16:46:19 2008 From: toby at etjohnson.us (Toby Johnson) Date: Mon Mar 10 16:46:23 2008 Subject: Infinite recursion in mergeparentdata In-Reply-To: <257413283.20080308125013@rhughes.net> References: <1565404365.20070724203840@rhughes.net> <46AA04F7.7080709@nogga.de> <257413283.20080308125013@rhughes.net> Message-ID: <47D59E1B.5060909@etjohnson.us> Richard Hughes wrote: > This is a resurrection of a thread from 6 months ago, for which I now > have more information. The basic background was: > > >>> I'm getting a crash during conversion when GetPathDepth recurses >>> itself forever (Perl exhausts its address space then segfaults). It >>> appears to be caused by a pair of physicalaction rows which refer to >>> each other >>> > > I've now put some effort into this and have created the following > (which are all at the bottom of this e-mail): > > 1. A minimal testcase, in the form of a batch file which should be run > on a brand new, totally empty sourcesafe database and which will > create something which crashes vss2svn (r336). > > 2. The entire contents of the PhysicalAction table resulting from the > above database, as it exists upon entry to MergeParentData(). > > 3. A patch against vss2svn.pl r336 which makes the problem go away. > I'm reluctant to use the word 'fixes' because I don't understand > enough about what the parentdata column is used for to be at all sure > that I've done the right thing. I have, however, confirmed that the > resultant svn repository has the correct history for my testcase. > > I'd be very appreciative if somebody knowledgable could review this > patch. It does still throw the warning "ERROR -- Multiple child recs > for parent MOVE rec '7' at vss2svn.pl line 740", but I believe this to > be harmless because the timestamp check shortly thereafter sorts it > all out correctly. > Thanks Richard, although I wasn't able to test the patch since I no longer have vss2svn installed on this box, it looks like a reasonable approach and I can't think of any cases where your additional check would cause an issue. I committed in r337. toby -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20080310/545340f1/attachment.html From ashu.ngm at gmail.com Wed Mar 26 03:58:24 2008 From: ashu.ngm at gmail.com (Ashwini N) Date: Wed Mar 26 03:58:32 2008 Subject: vss2svn migratin problem Message-ID: hi all, I am using vss2svn tool to convert my vss repository to svn. I get the following error: "Changing name of 'Microsoft.ApplicationBlocks.Excep' to ' Microsoft.ApplicationBl ocks.ExceptionManagement.dll.refresh' from name cache COMMITTING CACHE 'PhysicalAction' TO DATABASE SETTING TASK MERGEPARENTDATA SETTING STEP 0 TASK: MERGEPARENTDATA DBD::SQLite2::st execute failed: unable to open database file(1) at dbdimp.clin e 419 at vss2svn.pl line 590. DBD::SQLite2::st execute failed: unable to open database file(1) at dbdimp.clin e 419 at vss2svn.pl line 590. " I have made the settings to disable the status cache temporary for TortiseSVN. (in TortoiseSVN settings, Icon overlays, status cache group) But still I get the error... I have not removed any .dll or .exe files from my VSS repository ...Will this cause the problem? If I have to remove them then how to remove these from the repository... My repository is of 3GB size.. Can any one please help....... Thank you in advance... -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20080326/9f7077f4/attachment.html From ashu.ngm at gmail.com Wed Mar 26 04:38:50 2008 From: ashu.ngm at gmail.com (Ashwini N) Date: Wed Mar 26 04:38:59 2008 Subject: vss2svn migratin problem In-Reply-To: References: Message-ID: hi all, To add to this, when run the analyse tool on my database I get the following message: Analyze Visual SourceSafe Version 6.0 (Build 8169) Database analysis in progress @ 3/26/08; 1:58p. File "d:\vssbaseversion\version.dat" not found Analysis complete @ 3/26/08; 1:58p On 3/26/08, Ashwini N wrote: > > hi all, > > I am using vss2svn tool to convert my vss repository to svn. > > I get the following error: > > "Changing name of 'Microsoft.ApplicationBlocks.Excep' to ' > Microsoft.ApplicationBl > ocks.ExceptionManagement.dll.refresh' from name cache > > > COMMITTING CACHE 'PhysicalAction' TO DATABASE > > SETTING TASK MERGEPARENTDATA > > SETTING STEP 0 > TASK: MERGEPARENTDATA > DBD::SQLite2::st execute failed: unable to open database file(1) at > dbdimp.c lin > e 419 at vss2svn.pl line 590. > DBD::SQLite2::st execute failed: unable to open database file(1) at > dbdimp.c lin > e 419 at vss2svn.pl line 590. > > " > > > > I have made the settings to disable the status cache temporary for > TortiseSVN. (in TortoiseSVN settings, Icon overlays, status cache group) > But still I get the error... > I have not removed any .dll or .exe files from my VSS repository ...Will > this cause the problem? > If I have to remove them then how to remove these from the repository... > My repository is of 3GB size.. > > Can any one please help....... > > Thank you in advance... > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20080326/07b44292/attachment.html From cyreve at gmail.com Thu Mar 27 16:33:31 2008 From: cyreve at gmail.com (Richard Hughes) Date: Thu Mar 27 16:33:44 2008 Subject: Infinite recursion in mergeparentdata In-Reply-To: <47D59E1B.5060909@etjohnson.us> References: <1565404365.20070724203840@rhughes.net> <46AA04F7.7080709@nogga.de> <257413283.20080308125013@rhughes.net> <47D59E1B.5060909@etjohnson.us> Message-ID: <1793163583.20080327203331@rhughes.net> >> I'm getting a crash during conversion when GetPathDepth recurses >> itself forever (Perl exhausts its address space then segfaults). It >> appears to be caused by a pair of physicalaction rows which refer to >> each other > > Thanks Richard, although I wasn't able to test the patch since I > no longer have vss2svn installed on this box, it looks like a > reasonable approach and I can't think of any cases where your > additional check would cause an issue. I committed in r337. Looks like I still haven't got it quite right. It no longer crashes, but there was another move of one of those directories after the previous testcase which doesn't get coalesced in MergeMoveData because their parentdata values are different. This results in the import getting confused, converting the first MOVE (which was the MOVE_FROM) into a SHARE, and making the second MOVE (was MOVE_TO) a move into /orphaned because it doesn't have an info column. Things get progressively worse from there, because the directory structure is incorrect and so future commits fail (in svnadmin load). Looking through the vss2svn history, it looks like in r313 (the patch which overloaded parentdata to contain the depth of the tree, not just a boolean) the GetChildRecs function was never updated to ignore the non-boolean aspect and so doesn't find the MOVE_TO record to merge with because it's got a different nonzero parentdata. Anyway, as before, here's a testcase and proposed patch for your review. Richard. -----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<----- **** make-broken-db.bat setlocal set SSDIR=c:\vss\test\db ss create -YAdmin -I- $/first ss create -YAdmin -I- $/first/second ss create -YAdmin -I- $/third sleep 2 ss move -YAdmin $/first/second $/third/second sleep 2 ss move -YAdmin $/third/second $/first/second sleep 2 ss move -YAdmin $/third $/first/second/third sleep 2 ss move -YAdmin $/first/second/third $/third endlocal -----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<----- **** vss2svn.pl.r337.patch Index: vss2svn.pl =================================================================== --- vss2svn.pl (revision 337) +++ vss2svn.pl (working copy) @@ -655,6 +655,7 @@ # we don't get the wrong row. $parentdata = 0 unless defined $parentdata; + $parentdata = 1 if $parentdata != 0; my $sql = <<"EOSQL"; SELECT @@ -662,7 +663,7 @@ FROM PhysicalAction WHERE - parentdata = ? + MIN(parentdata, 1) = ? AND physname = ? AND actiontype = ? AND author = ?