From KelseyPJ at fairmountautomation.com Thu May 10 22:53:33 2007 From: KelseyPJ at fairmountautomation.com (Patrick J. Kelsey) Date: Thu May 10 22:53:39 2007 Subject: DumpFile.pm patch Message-ID: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local> Hello, I am having very good success in migrating a 310 MiB VSS database that has a number of 'destroyed' items and incorrect timestamps to svn, but only with the following patch to DumpFile.pm. The symptom that prompted me to make this change was a "file not found" error from svnadmin load when it encountered an entry in the dumpfile that was attempting to copy to a subdirectory in the orphaned directory whose parent directories had not yet been created. I haven't fully investigated the exact sequence of events that leads to this condition, since the intuitively simple (perhaps naive) patch below seems to have been quite successful for me. The gist of the change is this: if the parent path of the target path of a move operation does not exist, then create it (code largely copied from the add handler). I am still working on a thorough review of the results of the conversion, but so far a number of diffs of projects at head and at a few labels are coming up clean. Index: script/Vss2Svn/Dumpfile.pm =================================================================== --- script/Vss2Svn/Dumpfile.pm (revision 309) +++ script/Vss2Svn/Dumpfile.pm (working copy) @@ -408,6 +408,21 @@ return 0; } +### HACK + my $success = $self->{repository}->exists_parent ($itempath); + if(!defined($success)) { + $self->add_error("Path consistency failure while trying to move " + . "item '$oldpath' to '$itempath' at revision $data->{revision_id}; skipping"); + return 0; + } + elsif ($success == 0) { + $self->add_error("Parent path missing while trying to move " + . "item '$oldpath' to '$itempath' at revision $data->{revision_id}: adding missing " + . "parents"); + $self->_create_svn_path ($nodes, $itempath); + } +### ENDHACK + my $node = Vss2Svn::Dumpfile::Node->new(); $node->set_initial_props($itempath, $data); $node->{action} = 'add'; Regards, Patrick -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070510/6cc18585/attachment.html From stephen.lee at hexagonmetrology.com Fri May 11 06:09:44 2007 From: stephen.lee at hexagonmetrology.com (Stephen Lee) Date: Fri May 11 06:09:52 2007 Subject: DumpFile.pm patch In-Reply-To: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local> References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local> Message-ID: <464440E8.3090901@wilcoxassoc.com> Patrick J. Kelsey wrote: > I am having very good success in migrating a 310 MiB VSS database that > has a number of 'destroyed' items and incorrect timestamps to svn, but > only with the following patch to DumpFile.pm. You seem to have independently come up with the same patch (except for the exact error message) as the DumpFile.pm section of my patch from February. http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/2007-February/000927.html contains patch against revision 285 http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/2007-February/001022.html contains patch against revision 300 Although I probably confused it somewhat by talking about several other things in the same email! Perhaps worth someone with access checking in at least the DumpFile.pm part of the patch... -- Stephen Lee Software Engineer, Vision Group - Pro-Measure Leader Wilcox Associates Inc. (U.K.) From KelseyPJ at fairmountautomation.com Fri May 11 08:02:35 2007 From: KelseyPJ at fairmountautomation.com (Patrick J. Kelsey) Date: Fri May 11 08:05:56 2007 Subject: DumpFile.pm patch References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local> <464440E8.3090901@wilcoxassoc.com> Message-ID: <98C43FB2347B7744ABCEA2971F6D5063090743@ddg1000.FairmountAutomation.local> Ah, I should have searched the list for patches to DumpFile.pm after I concocted this one. Well you can certainly have all the glory :) and consider my report another vote for integration of your patch. Patrick ________________________________ From: vss2svn-users-bounces@lists.pumacode.org on behalf of Stephen Lee Sent: Fri 5/11/2007 6:09 AM To: Vss2Svn Users Subject: Re: DumpFile.pm patch Patrick J. Kelsey wrote: > I am having very good success in migrating a 310 MiB VSS database that > has a number of 'destroyed' items and incorrect timestamps to svn, but > only with the following patch to DumpFile.pm. You seem to have independently come up with the same patch (except for the exact error message) as the DumpFile.pm section of my patch from February. http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/2007-February/000927.html contains patch against revision 285 http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/2007-February/001022.html contains patch against revision 300 Although I probably confused it somewhat by talking about several other things in the same email! Perhaps worth someone with access checking in at least the DumpFile.pm part of the patch... -- Stephen Lee Software Engineer, Vision Group - Pro-Measure Leader Wilcox Associates Inc. (U.K.) _______________________________________________ vss2svn-users mailing list Project homepage: http://www.pumacode.org/projects/vss2svn/ Subscribe/Unsubscribe/Admin: http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org Mailing list web interface (with searchable archives): http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 4914 bytes Desc: not available Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070511/801d2e85/attachment.bin From stephen.lee at hexagonmetrology.com Fri May 11 08:18:16 2007 From: stephen.lee at hexagonmetrology.com (Stephen Lee) Date: Fri May 11 08:18:25 2007 Subject: DumpFile.pm patch In-Reply-To: <98C43FB2347B7744ABCEA2971F6D5063090743@ddg1000.FairmountAutomation.local> References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local> <464440E8.3090901@wilcoxassoc.com> <98C43FB2347B7744ABCEA2971F6D5063090743@ddg1000.FairmountAutomation.local> Message-ID: <46445F08.8000009@wilcoxassoc.com> Patrick J. Kelsey wrote: > Ah, I should have searched the list for patches to DumpFile.pm after I > concocted this one. Well you can certainly have all the glory :) and > consider my report another vote for integration of your patch. Don't particularly care for glory... but kinda recognised the code and realised the similarity... and figured that one of our patches to DumpFile.pm might count as a code review of the other! ;) -- Stephen Lee Software Engineer, Vision Group - Pro-Measure Leader Wilcox Associates Inc. (U.K.) From toby at etjohnson.us Fri May 11 13:56:31 2007 From: toby at etjohnson.us (Toby Johnson) Date: Fri May 11 13:55:46 2007 Subject: Patches and tickets (was: DumpFile.pm patch) In-Reply-To: <464440E8.3090901@wilcoxassoc.com> References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local> <464440E8.3090901@wilcoxassoc.com> Message-ID: <4644AE4F.5090709@etjohnson.us> Stephen Lee wrote: > Patrick J. Kelsey wrote: >> I am having very good success in migrating a 310 MiB VSS database >> that has a number of 'destroyed' items and incorrect timestamps to >> svn, but only with the following patch to DumpFile.pm. > You seem to have independently come up with the same patch (except for > the exact error message) as the DumpFile.pm section of my patch from > February. > > http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/2007-February/000927.html > > contains patch against revision 285 > > http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/2007-February/001022.html > > contains patch against revision 300 > > > Although I probably confused it somewhat by talking about several > other things in the same email! > > > Perhaps worth someone with access checking in at least the DumpFile.pm > part of the patch... > Thanks for the patch guys; I guess if two people independently submit the same patch then I can assume it works! I've committed this in r310. Stephen, if you have any other suggested patches which were not committed and which don't exist as an open Ticket (http://www.pumacode.org/projects/vss2svn/report/1), please feel free to refresh my memory (I know you were discussing several issues w/ Dirk at the time and I'm not familiar w/ that whole conversation). I can give you or others access to create/modify tickets directly if you'd like (I had this turned on globally but was getting ticket spam, which Trac makes very hard to delete). Since myself and the handful of committers for this project don't have much time to work on this anymore, perhaps we need to come up with a better way for submitted patches to be accepted. I obviously don't want to just commit every patch that is submitted via email, but I certainly don't want to lose track of the good ones either. Thoughts on how to do this are welcome. toby From Aaron.Larson at honeywell.com Sun May 13 00:24:21 2007 From: Aaron.Larson at honeywell.com (Larson, Aaron (SWCOE)) Date: Sun May 13 00:24:29 2007 Subject: Patches and tickets (was: DumpFile.pm patch) In-Reply-To: <4644AE4F.5090709@etjohnson.us> References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local><464440E8.3090901@wilcoxassoc.com> <4644AE4F.5090709@etjohnson.us> Message-ID: >>>>> "TJ" == Toby writes: TJ> Thanks for the patch guys; I guess if two people independently TJ> submit the same patch then I can assume it works! I've committed TJ> this in r310. The description of this patch certainly sounds like the problem I've been having with my databases as well. Any chance a nightly build could be done that includes the patch? TJ> Since myself and the handful of committers for this project don't TJ> have much time to work on this anymore, perhaps we need to come up TJ> with a better way for submitted patches to be accepted. I wish I could help. For whatever reason, when I run from source, the results are very different than the nightly build, otherwise I'd volunteer as a tester. From toby at etjohnson.us Mon May 14 22:02:25 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon May 14 22:02:35 2007 Subject: Patches and tickets In-Reply-To: References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local><464440E8.3090901@wilcoxassoc.com> <4644AE4F.5090709@etjohnson.us> Message-ID: <464914B1.8040209@etjohnson.us> Larson, Aaron (SWCOE) wrote: >>>>>> "TJ" == Toby writes: >>>>>> > > TJ> Thanks for the patch guys; I guess if two people independently > TJ> submit the same patch then I can assume it works! I've committed > TJ> this in r310. > > The description of this patch certainly sounds like the problem I've > been having with my databases as well. Any chance a nightly build > could be done that includes the patch? > A new nightly build has been uploaded with this change. toby From Aaron.Larson at honeywell.com Tue May 15 13:40:23 2007 From: Aaron.Larson at honeywell.com (Larson, Aaron (SWCOE)) Date: Tue May 15 13:40:33 2007 Subject: Patches and tickets In-Reply-To: <464914B1.8040209@etjohnson.us> References: <98C43FB2347B7744ABCEA2971F6D5063090741@ddg1000.FairmountAutomation.local><464440E8.3090901@wilcoxassoc.com> <4644AE4F.5090709@etjohnson.us> <464914B1.8040209@etjohnson.us> Message-ID: >>>>> "TJ" == Toby writes: AL> Larson, Aaron (SWCOE) wrote: AL> The description of this patch certainly sounds like the problem AL> I've been having with my databases as well. Any chance a nightly AL> build could be done that includes the patch? TJ> A new nightly build has been uploaded with this change. Thanks. Sadly, it didn't fix my problem. :-( From anogin at hrl.com Thu May 17 17:47:32 2007 From: anogin at hrl.com (Aleksey Nogin) Date: Thu May 17 17:47:36 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? Message-ID: <464CCD74.6020301@hrl.com> If I use the vss2svn (latest HEAD version) with the --auto_props option and the auto-props file sets svn:eol-style=native one some of the files, the files would not be converted correctly. The problem here is that the dump file is _required_ to only have the LF line terminators on files marked with svn:eol-style=native (see http://svn.haxx.se/dev/archive-2006-07/0981.shtml ), but vss2svn does not make any attempt to do that... Should I file an issue report on this? Aleksey From finnied at aciworldwide.com Thu May 17 22:44:46 2007 From: finnied at aciworldwide.com (finnied@aciworldwide.com) Date: Thu May 17 22:44:54 2007 Subject: svnadmin: File not found In-Reply-To: <4608DB18.3060905@nogga.de> Message-ID: Dirk & All, I came up with, and coded a solution for this problem. The patch is supplied below, if anyone is interested. To summarise, this is to fix the "svnadmin load" problem where it emits the following error: vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv svnadmin: File not found: transaction '20678-1', path 'orphaned/_YEDAAAAA/S01/InstallShield/WebGate/Setup Files/Uncompressed Files/Language Independent/OS Independent/setup.bmp' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The main problem stemmed from the following query in the BUILDACTIONHIST phase: my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' . 'itemtype ASC, priority ASC, sortkey ASC, action_id ASC'; If 2 or more projects were added at the exact same time, then depending on their physical VSS name, they may be returned by this query in the wrong order (i.e. ADDing a subproject before its parent project has been ADDed). To fix this, I decided to extend the use of "parentdata" in the PhysicalAction table, and instead treat it like a project "depth" attribute. During the MERGEPARENTDATA phase, all of the parent records are updated with their project path depth as an integer, such that the depth increases the further a project/file gets from the root project. The query is then changed to: my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' . 'itemtype ASC, priority ASC, parentdata ASC, sortkey ASC, action_id ASC'; The solution ensures that subprojects will have a parentdata value greater than that of their parent project, and therefore be returned by this query after their parent project. This fixed the problem for me. If anyone can think of a better solution, go for it. Delving into this problem has made me truly realise how badly designed VSS is ! A disclaimer - PERL is not my forte, so if there is a cleverer way of doing it, I'd be happy for people to edit my efforts :-) Dave Index: script/vss2svn.pl =================================================================== --- script/vss2svn.pl (revision 311) +++ script/vss2svn.pl (working copy) @@ -526,13 +526,13 @@ my($sth, $rows, $row); $sth = $gCfg{dbh}->prepare('SELECT * FROM PhysicalAction ' - . 'WHERE parentdata = 1'); + . 'WHERE parentdata > 0'); $sth->execute(); # need to pull in all recs at once, since we'll be updating/deleting data $rows = $sth->fetchall_arrayref( {} ); - my($childrecs, $child, $id); + my($childrecs, $child, $id, $depth); my @delchild = (); foreach $row (@$rows) { @@ -543,6 +543,8 @@ . "'$row->{action_id}'"); } + $depth = &GetPathDepth($row); + foreach $child (@$childrecs) { &UpdateParentRec($row, $child); push(@delchild, $child->{action_id}); @@ -558,6 +560,72 @@ } # End MergeParentData ############################################################################### +# GetPathDepth +############################################################################### +sub GetPathDepth { + my($row) = @_; + + # If we've already worked out the depth of this row, return it immediately + if ($row->{parentdata} > 1) { + return $row->{parentdata}; + } + + my($maxParentDepth, $depth, $parents, $parent); + + # Get the row(s) corresponding to the parent(s) of this row, and work out + # the maximum depth + + my $sql = <<"EOSQL"; +SELECT + * +FROM + PhysicalAction +WHERE + parentdata > 0 + AND physname = ? + AND actiontype = ? +EOSQL + + my $sth = $gCfg{dbh}->prepare($sql); + $sth->execute( @{ $row }{qw(parentphys actiontype)} ); + + $parents = $sth->fetchall_arrayref( {} ); + $maxParentDepth = 0; + foreach $parent (@$parents) { + $depth = &GetPathDepth($parent); + $maxParentDepth = ($depth > $maxParentDepth) ? $depth : $maxParentDepth; + } + + # Depth of this path becomes one more than the maximum parent depth + $depth = $maxParentDepth + 1; + + # Update the row for this record + &UpdateDepth($row, $depth); + + return $depth; +} # End GetPathDepth + +############################################################################### +# UpdateDepth +############################################################################### +sub UpdateDepth { + my($row, $depth) = @_; + + my $sql = <<"EOSQL"; +UPDATE + PhysicalAction +SET + parentdata = ? +WHERE + action_id = ? +EOSQL + + my $sth = $gCfg{dbh}->prepare($sql); + $sth->execute( $depth, $row->{action_id} ); + +} # End UpdateParentRec + +############################################################################### # GetChildRecs ############################################################################### sub GetChildRecs { @@ -739,7 +807,7 @@ my($sth, $row, $action, $handler, $physinfo, $itempaths, $allitempaths); my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' - . 'itemtype ASC, priority ASC, sortkey ASC, action_id ASC'; + . 'itemtype ASC, priority ASC, parentdata ASC, sortkey ASC, action_id ASC'; $sth = $gCfg{dbh}->prepare($sql); $sth->execute(); This e-mail message and any attachments may contain confidential, proprietary or non-public information. This information is intended solely for the designated recipient(s). If an addressing or transmission error has misdirected this e-mail, please notify the sender immediately and destroy this e-mail. Any review, dissemination, use or reliance upon this information by unintended recipients is prohibited. Any opinions expressed in this e-mail are those of the author personally. Dirk Sent by: vss2svn-users-bounces@lists.pumacode.org 27-Mar-2007 06:51 PM Please respond to Vss2Svn Users To Vss2Svn Users cc Subject Re: Fw: ssphys error: failed to read necessary amount of data from input file Dirk schrieb: > >> 86197 TUADAAAA EDADAAAA 1 ADD >> 1 0 \N >> 86198 LUADAAAA TUADAAAA 1 ADD >> /orphaned/_YEDAAAAA/S01/InstallShield/WebGate/Setup >> Files/Uncompressed Files/Language Independent/ 1 >> 0 \N Ok, the problem is, that you managed to add a subproject to a project at exactly the same timestamp as you added the project to its parent project. vss2svn is confused about this fact, and as a result it first adds the subproject, then the project itself. The above two actions must actually be reversed. Here are the corresponding lines from your physical history without any confidential information > 58593 TUADAAAA \N LUADAAAA ADD Language Independent/ 1 1139237015 xxxx 0 \N 1 AAAADAUT 1 \N \N > 98983 EDADAAAA \N TUADAAAA ADD OS Independent/ 1 1139237015 xxxx 0 \N 1 AAAADADE 1 \N \N TUADAAAA is added to LUADAAAA at timestamp 1139237015 . EDADAAAA is added to TUADAAAA also at timestamp 1139237015 . Gimme some time to think about a solution. Dirk _______________________________________________ vss2svn-users mailing list Project homepage: http://www.pumacode.org/projects/vss2svn/ Subscribe/Unsubscribe/Admin: http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org Mailing list web interface (with searchable archives): http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070518/97354421/attachment.html From amos.shapira at gmail.com Tue May 22 02:11:34 2007 From: amos.shapira at gmail.com (Amos Shapira) Date: Tue May 22 02:11:47 2007 Subject: still running out of memory using r311 Message-ID: <9c2cca270705212311q2161f61cx82395dda4027d81b@mail.gmail.com> Hi, I've checked out vss2svn r311 and built it on Debian Etch and it executed pretty well according to the instructions but still bombs on one of the files with an "out of memory" error. I tried multiple times. The last run was executed with the --debug flag but that didn't seem to generate any more useful output: "ssphys" get -b -v1 --force-overwrite -ewindows-1252 "/tmp/vss2svn/VSS//data/h/hfmaaaaa" ./_vss2svn/vssdata/HF/HFMAAAAA Out of memory! What else can I do with it? Thanks, --Amos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070522/cd172984/attachment.html From amos.shapira at gmail.com Tue May 22 19:25:42 2007 From: amos.shapira at gmail.com (Amos Shapira) Date: Tue May 22 19:25:46 2007 Subject: removing files from input VSS repository Message-ID: <9c2cca270705221625r408cda71j2c309e7f6a4c71c1@mail.gmail.com> Hello, In continuation to my previous question about the vss2svn.pl script running out of memory - as far as I can see, this happens on a huge binary input file (803Mb). I wonder how can I make the script skip this file since I don't actually care much about moving it into SVN. Would it be skipped if I deleted it through the VSS interface? Is there another way to skip such files? Thanks, --Amos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070523/57de38ce/attachment.html From Aaron.Larson at honeywell.com Wed May 23 00:11:51 2007 From: Aaron.Larson at honeywell.com (Larson, Aaron (SWCOE)) Date: Wed May 23 00:12:00 2007 Subject: removing files from input VSS repository In-Reply-To: <9c2cca270705221625r408cda71j2c309e7f6a4c71c1@mail.gmail.com> References: <9c2cca270705221625r408cda71j2c309e7f6a4c71c1@mail.gmail.com> Message-ID: >>>>> "AS" == Amos writes: AS> In continuation to my previous question about the vss2svn.pl AS> script running out of memory - as far as I can see, this happens AS> on a huge binary input file (803Mb). I wonder how can I make the AS> script skip this file since I don't actually care much about AS> moving it into SVN. I ran into a similar problem and I just replaced the file with a README. A caveat however is that my translation never did succeed, but I don't think it was due to the changed file. Something like this: for f in $(find . -size +300M); do mv $f $f.save echo "Large file replaced with this readme to satisfy vss2svn conversion script." > $f Done I'm a complete vss noob though, so I wouldn't suggest taking my word for it... From amos.shapira at gmail.com Wed May 23 01:47:08 2007 From: amos.shapira at gmail.com (Amos Shapira) Date: Wed May 23 01:47:13 2007 Subject: removing files from input VSS repository In-Reply-To: References: <9c2cca270705221625r408cda71j2c309e7f6a4c71c1@mail.gmail.com> Message-ID: <9c2cca270705222247y357383fcl497c8fb7172e17bb@mail.gmail.com> On 23/05/07, Larson, Aaron (SWCOE) wrote: > > >>>>> "AS" == Amos writes: > > AS> In continuation to my previous question about the vss2svn.pl > AS> script running out of memory - as far as I can see, this happens > AS> on a huge binary input file (803Mb). I wonder how can I make the > AS> script skip this file since I don't actually care much about > AS> moving it into SVN. > > I ran into a similar problem and I just replaced the file with a > README. A caveat however is that my translation never did succeed, > but I don't think it was due to the changed file. Something like > this: > > for f in $(find . -size +300M); do > mv $f $f.save > echo "Large file replaced with this readme to satisfy vss2svn > conversion script." > $f > Done > > I'm a complete vss noob though, so I wouldn't suggest taking my word for > it... Thanks. I'm a complete VSS noob too. Today I talked to someone who knows it better than me and he told me how I can completely remove files from within VSS itself. I'm now on the last of multiple runs of analyze.exe with -F, -C and -D before copying over the files to Linux for another attempt to run vss2svn.pl My tactic so far was to find and remove all .exe, .dll and .bmp files (and some other large binary setup files) from inside VSS. These are non-versioned anyway and were put there more as a backup thing. Once the conversion is over I expect I'll be able to just checkin the last copy of these files directly into SVN. Cheers, --Amos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070523/0a8fbd09/attachment.html From guillaume at scio.com.mx Thu May 24 16:15:00 2007 From: guillaume at scio.com.mx (Guillaume Gros) Date: Thu May 24 16:16:58 2007 Subject: Some Comments Message-ID: <4655F244.5020008@scio.com.mx> I've recently tried to convert a VSS repository to Subversion. We have all our projects in a single VSS repository (one per folder). So I first tried to convert the whole repository and use filtering to only insert entries related to the project I wanted to convert. (method recommended in the Wiki) This project had shares that were deleted and various files ended in the orphaned folder. So when I tried to svndumpfilter for only the project folder I was getting errors when svn was trying to copy from orphaned back to the project folder.(missing file error) So I thought I would include the orphaned folder but then again ran into trouble because orphaned was trying to copy some files to another project that was not included in the filter.... To solve that I ended up searching for all copy operations from orphaned to my project folder and with that created a big include that would include all the subfolders that were needed by a svn copy/move operation ( orphaned/YYYYAAA ..etc) After doing that svndumpfilter managed to create a file. I inserted that file into SVN and tried to compare and realized that some files were missing (actually whole folders too) I searched the dumpfile and realized that those missing files and missing folders were in the orphaned folder and were never moved to the project folder.... At that point I was about to give up and insert a latest copy into SVN (effectively losing all history) and then tried to do a backup of the project restored that backup into a fresh new VSS repository and ran the conversion again (actually this time I used the latest nightly). At the end I inserted the content of the dump file and the project folder was not even there. The whole project was in orphaned. But I just moved/renamed and then compared. Only very few files were missing. Those missing files were all files that were created and never modified (i.e. no history) so I added those missing files manually and ended up with a complete SVN repository with full history. I am not sure if the backup/restore is better than svndumpfilter but it surely is a lot easier/faster. The first try was made using the recommended alpha version while the second was made using the latest nightly so maybe the latest nightly would have managed to move missing files from orphaned on its own using the original repository but who knows... Anyway hats off to the developers, that project folder was the worst so I guess if vss2svn managed to convert this one it should be able to convert all others..... From finnied at aciworldwide.com Thu May 24 17:27:20 2007 From: finnied at aciworldwide.com (finnied@aciworldwide.com) Date: Thu May 24 17:27:31 2007 Subject: Some Comments In-Reply-To: <4655F244.5020008@scio.com.mx> Message-ID: An interesting experience. Unfortunately I'm now about to abort my efforts with vss2svn. After finding and fixing a problem that prevented the dumpfile from being loaded into subversion, a comparison of the HEAD revision of the Subversion database against a fresh "Get Latest" from VSS showed hundreds of different and/or missing files. There were many many orphaned directories, and no doubt the missing files were buried in there somewhere, but I don't have time to look for all of them. I don't know why some of the files would be different, though. What would concern me the most, however, is that if the HEAD revision does not match, can I be sure about the history that I am painstakingly trying to preserve ? So, I agree, vss2svn is a great tool. However, methinks our VSS database is perhaps so corrupt that it was too hard a task to convert it. Dave This e-mail message and any attachments may contain confidential, proprietary or non-public information. This information is intended solely for the designated recipient(s). If an addressing or transmission error has misdirected this e-mail, please notify the sender immediately and destroy this e-mail. Any review, dissemination, use or reliance upon this information by unintended recipients is prohibited. Any opinions expressed in this e-mail are those of the author personally. Guillaume Gros Sent by: vss2svn-users-bounces@lists.pumacode.org 25-May-2007 06:15 AM Please respond to Vss2Svn Users To vss2svn-users@lists.pumacode.org cc Subject Some Comments I've recently tried to convert a VSS repository to Subversion. We have all our projects in a single VSS repository (one per folder). So I first tried to convert the whole repository and use filtering to only insert entries related to the project I wanted to convert. (method recommended in the Wiki) This project had shares that were deleted and various files ended in the orphaned folder. So when I tried to svndumpfilter for only the project folder I was getting errors when svn was trying to copy from orphaned back to the project folder.(missing file error) So I thought I would include the orphaned folder but then again ran into trouble because orphaned was trying to copy some files to another project that was not included in the filter.... To solve that I ended up searching for all copy operations from orphaned to my project folder and with that created a big include that would include all the subfolders that were needed by a svn copy/move operation ( orphaned/YYYYAAA ..etc) After doing that svndumpfilter managed to create a file. I inserted that file into SVN and tried to compare and realized that some files were missing (actually whole folders too) I searched the dumpfile and realized that those missing files and missing folders were in the orphaned folder and were never moved to the project folder.... At that point I was about to give up and insert a latest copy into SVN (effectively losing all history) and then tried to do a backup of the project restored that backup into a fresh new VSS repository and ran the conversion again (actually this time I used the latest nightly). At the end I inserted the content of the dump file and the project folder was not even there. The whole project was in orphaned. But I just moved/renamed and then compared. Only very few files were missing. Those missing files were all files that were created and never modified (i.e. no history) so I added those missing files manually and ended up with a complete SVN repository with full history. I am not sure if the backup/restore is better than svndumpfilter but it surely is a lot easier/faster. The first try was made using the recommended alpha version while the second was made using the latest nightly so maybe the latest nightly would have managed to move missing files from orphaned on its own using the original repository but who knows... Anyway hats off to the developers, that project folder was the worst so I guess if vss2svn managed to convert this one it should be able to convert all others..... _______________________________________________ vss2svn-users mailing list Project homepage: http://www.pumacode.org/projects/vss2svn/ Subscribe/Unsubscribe/Admin: http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org Mailing list web interface (with searchable archives): http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070524/8f7656e6/attachment.html From toby at etjohnson.us Thu May 24 17:51:01 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu May 24 17:50:00 2007 Subject: Some Comments In-Reply-To: References: Message-ID: <465608C5.2010200@etjohnson.us> Hello Guillaume and Dave, thanks for sending your experiences. As we have found from numerous user experiences, every case seems to be different because VSS finds unique and interesting ways to corrupt itself. :) Most people find that creating and restoring a backup prior to conversion will actually make the process more difficult. This is due to the way VSS creates the timestamps when a restore is performed; it appears that the contents of a folder were created after the folder itself so there is no good way of determining where those files should go. At least in this case it sounds like the history itself was maintained even though they were considered "orphaned", so that's a good thing. Dave, even if you aren't able to get a near-perfect conversion, another option is to convert as much as possible then overlay the "get latest" on top and commit that. Another option that some end up using when they can't get a good conversion is to just keep the VSS repo around in read-only mode and start fresh in SVN. It may not be your preferred solution and I understand the frustration in not being able to get a good conversion, but at least from personal experience I can say that when we converted over, we ended up needing the "old" history far less than we thought we would. So hopefully the inconvenience is only temporary until you've been using SVN for a while. Good luck, toby From finnied at aciworldwide.com Thu May 24 17:58:11 2007 From: finnied at aciworldwide.com (finnied@aciworldwide.com) Date: Thu May 24 17:58:21 2007 Subject: Some Comments In-Reply-To: <465608C5.2010200@etjohnson.us> Message-ID: Thanks Toby. Agreed re. keeping the VSS database around in read-only mode. That is the approach that I have chosen. Re. the RESTORE issue - I noticed that there is physical information indicating when the restore is done. I think vss2svn could be made to "back-track" through the parent projects starting from each restore, and adjust their timestamps - or some other similar solution. However, I think the problems with my VSS database are bigger than that, so I haven't spent any more time on this solution. Dave This e-mail message and any attachments may contain confidential, proprietary or non-public information. This information is intended solely for the designated recipient(s). If an addressing or transmission error has misdirected this e-mail, please notify the sender immediately and destroy this e-mail. Any review, dissemination, use or reliance upon this information by unintended recipients is prohibited. Any opinions expressed in this e-mail are those of the author personally. Toby Johnson Sent by: vss2svn-users-bounces@lists.pumacode.org 25-May-2007 07:51 AM Please respond to Vss2Svn Users To Vss2Svn Users cc Subject Re: Some Comments Hello Guillaume and Dave, thanks for sending your experiences. As we have found from numerous user experiences, every case seems to be different because VSS finds unique and interesting ways to corrupt itself. :) Most people find that creating and restoring a backup prior to conversion will actually make the process more difficult. This is due to the way VSS creates the timestamps when a restore is performed; it appears that the contents of a folder were created after the folder itself so there is no good way of determining where those files should go. At least in this case it sounds like the history itself was maintained even though they were considered "orphaned", so that's a good thing. Dave, even if you aren't able to get a near-perfect conversion, another option is to convert as much as possible then overlay the "get latest" on top and commit that. Another option that some end up using when they can't get a good conversion is to just keep the VSS repo around in read-only mode and start fresh in SVN. It may not be your preferred solution and I understand the frustration in not being able to get a good conversion, but at least from personal experience I can say that when we converted over, we ended up needing the "old" history far less than we thought we would. So hopefully the inconvenience is only temporary until you've been using SVN for a while. Good luck, toby _______________________________________________ vss2svn-users mailing list Project homepage: http://www.pumacode.org/projects/vss2svn/ Subscribe/Unsubscribe/Admin: http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org Mailing list web interface (with searchable archives): http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070525/2469bde6/attachment-0001.html From Jonathan.Perret at augure.com Fri May 25 11:09:36 2007 From: Jonathan.Perret at augure.com (Jonathan Perret) Date: Fri May 25 11:10:10 2007 Subject: Some Comments In-Reply-To: <465608C5.2010200@etjohnson.us> References: <465608C5.2010200@etjohnson.us> Message-ID: Toby Johnson wrote: > Dave, even if you aren't able to get a near-perfect conversion, another > option is to convert as much as possible then overlay the "get latest" > on top and commit that. Another option that some end up using when they > can't get a good conversion is to just keep the VSS repo around in > read-only mode and start fresh in SVN. Hello Toby, I would like to point out a third, intermediate, option that I ended up using : convert as much as possible into a *read-only* SVN repository, then do all new work in another, fresh, SVN repository. This gets us all the benefits of SVN for our historical data (better performance, "atomic" commits, "blame" functionality), while letting us define a clean new repository for new work. The only drawback is that when searching the history you have to switch repositories when you hit the cut-off point but in practice that's not such a big deal. Cheers, --Jonathan From toby at etjohnson.us Fri May 25 14:02:50 2007 From: toby at etjohnson.us (Toby Johnson) Date: Fri May 25 14:01:51 2007 Subject: Some Comments In-Reply-To: References: Message-ID: <465724CA.6000208@etjohnson.us> finnied@aciworldwide.com wrote: > > Thanks Toby. > > Agreed re. keeping the VSS database around in read-only mode. That is > the approach that I have chosen. > > Re. the RESTORE issue - I noticed that there is physical information > indicating when the restore is done. I think vss2svn could be made to > "back-track" through the parent projects starting from each restore, > and adjust their timestamps - or some other similar solution. However, > I think the problems with my VSS database are bigger than that, so I > haven't spent any more time on this solution. Yes, that is the approach we would probably take if we were to attempt fixing the RESTORE issue but realistically that issue is probably too big for any of the current developers to spend time on. toby From toby at etjohnson.us Fri May 25 14:07:24 2007 From: toby at etjohnson.us (Toby Johnson) Date: Fri May 25 14:06:25 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <464CCD74.6020301@hrl.com> References: <464CCD74.6020301@hrl.com> Message-ID: <465725DC.9000102@etjohnson.us> Aleksey Nogin wrote: > If I use the vss2svn (latest HEAD version) with the --auto_props > option and the auto-props file sets svn:eol-style=native one some of > the files, the files would not be converted correctly. The problem > here is that the dump file is _required_ to only have the LF line > terminators on files marked with svn:eol-style=native (see > http://svn.haxx.se/dev/archive-2006-07/0981.shtml ), but vss2svn does > not make any attempt to do that... > > Should I file an issue report on this? > > Aleksey Thanks Aleksy; I have disabled ticket submission by non-committers due to ticket spam but have filed your issue as ticket 51 (http://www.pumacode.org/projects/vss2svn/ticket/51). If you end up getting code written to handle this we would appreciate a patch! toby