From hyugaricdeau at gmail.com Mon Jun 11 17:47:29 2007 From: hyugaricdeau at gmail.com (Erik Bray) Date: Mon Jun 11 17:47:33 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors Message-ID: Greetings, First some background: I've been working on converting a rather large (~19 GB) VSS repository to SVN. Yes, it's a mess. The source repository is all full of gunk. But the other issues can be addressed separately. The issue here is that being such a large repository (the resulting SVN repository is about 22000 revisions), and considering that there are a few files in this repository of several hundred megabytes in size (why, I don't know), memory usage became a huge problem. Note that I've never used VSS in my life, and my background in Perl is fairly limited. I'm just the poor sucker who got tasked with converting this repository. This patch implements something more or less like the recommendation made by Toby here: http://thread.gmane.org/gmane.comp.version-control.subversion.vss2svn.user/1652/focus=1654 It adds a 'file' attribute to the Node class associating a node with the physical file that that node's contents are read from. It also keeps the 'text' attribute around, though now nothing is using it. In Dumpfile.pm it changes get_export_contents to get_export_file which sets the 'file' attribute, but doesn't actually open the file yet. Finally, in output_content, if $node->{text} is undefined, it checks for $node->{file}, and if that exists it uses File::Copy to copy that file's contents into the dump file. I've tested this with a couple smaller repositories, and our huge one, and it works as advertised. Whereas previously my memory usage would slowly increase, slowing my system to a crawl, now it just stays flat since File::Copy only buffers 2MB at a time. So assuming I haven't made any glaring errors, (and please point them out if I have), enjoy :) Erik -------------- next part -------------- A non-text attachment was scrubbed... Name: vss2svn-OOM-fix.patch Type: text/x-patch Size: 3742 bytes Desc: not available Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070611/e9e25c8a/vss2svn-OOM-fix.bin From toby at etjohnson.us Mon Jun 11 22:39:02 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jun 11 22:39:05 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors In-Reply-To: References: Message-ID: <466E0746.20005@etjohnson.us> Erik Bray wrote: > Greetings, > First some background: I've been working on converting a rather large > (~19 GB) VSS repository to SVN. Yes, it's a mess. The source > repository is all full of gunk. But the other issues can be addressed > separately. > > The issue here is that being such a large repository (the resulting > SVN repository is about 22000 revisions), and considering that there > are a few files in this repository of several hundred megabytes in > size (why, I don't know), memory usage became a huge problem. > > Note that I've never used VSS in my life, and my background in Perl is > fairly limited. I'm just the poor sucker who got tasked with > converting this repository. This patch implements something more or > less like the recommendation made by Toby here: > http://thread.gmane.org/gmane.comp.version-control.subversion.vss2svn.user/1652/focus=1654 > > > It adds a 'file' attribute to the Node class associating a node with > the physical file that that node's contents are read from. It also > keeps the 'text' attribute around, though now nothing is using it. > > In Dumpfile.pm it changes get_export_contents to get_export_file which > sets the 'file' attribute, but doesn't actually open the file yet. > Finally, in output_content, if $node->{text} is undefined, it checks > for $node->{file}, and if that exists it uses File::Copy to copy that > file's contents into the dump file. > > I've tested this with a couple smaller repositories, and our huge one, > and it works as advertised. Whereas previously my memory usage would > slowly increase, slowing my system to a crawl, now it just stays flat > since File::Copy only buffers 2MB at a time. So assuming I haven't > made any glaring errors, (and please point them out if I have), enjoy > :) Looks great Erik, thanks! I don't have time (or a VSS repo) to test this at the moment but the patch looks good and syntax checks OK so I went ahead and committed in r312. If anyone has problems with the resulting nightly build (or confirmation that it works correctly), let me know... toby From toby at etjohnson.us Mon Jun 11 22:53:03 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jun 11 22:53:05 2007 Subject: svnadmin: File not found In-Reply-To: References: Message-ID: <466E0A8F.2020404@etjohnson.us> finnied@aciworldwide.com wrote: > > Dirk & All, > > I came up with, and coded a solution for this problem. The patch is > supplied below, if anyone is interested. > > To summarise, this is to fix the "svnadmin load" problem where it > emits the following error: > > vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv > > > svnadmin: File not found: transaction '20678-1', path > 'orphaned/_YEDAAAAA/S01/InstallShield/WebGate/Setup Files/Uncompressed > Files/Language Independent/OS Independent/setup.bmp' > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > The main problem stemmed from the following query in the > BUILDACTIONHIST phase: > > my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' > . 'itemtype ASC, priority ASC, sortkey ASC, action_id ASC'; > > If 2 or more projects were added at the exact same time, then > depending on their physical VSS name, they may be returned by this > query in the wrong order (i.e. ADDing a subproject before its parent > project has been ADDed). > > To fix this, I decided to extend the use of "parentdata" in the > PhysicalAction table, and instead treat it like a project "depth" > attribute. During the MERGEPARENTDATA phase, all of the parent records > are updated with their project path depth as an integer, such that the > depth increases the further a project/file gets from the root project. > > The query is then changed to: > > my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' > . 'itemtype ASC, priority ASC, parentdata ASC, sortkey > ASC, action_id ASC'; > > The solution ensures that subprojects will have a parentdata value > greater than that of their parent project, and therefore be returned > by this query after their parent project. > > This fixed the problem for me. If anyone can think of a better > solution, go for it. Delving into this problem has made me truly > realise how badly designed VSS is ! > > A disclaimer - PERL is not my forte, so if there is a cleverer way of > doing it, I'd be happy for people to edit my efforts :-) > > Dave > Hi Dave, sorry it took so long to get back to you... as with the previous patch, I don't have time to test this (and it's not an error I personally ran into) but it looks logical at first glance so I went ahead and committed in r313; if it breaks anyone else's conversion hopefully we'll hear about it! However, I notice that you are not doing anything with the return value of &GetPathDepth() on line 546... it seems maybe that you at first used a return value then later decided to store it in the database instead? thanks, toby toby From toby at etjohnson.us Mon Jun 11 22:58:55 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jun 11 22:58:58 2007 Subject: Automated conversion script In-Reply-To: <1286330289.20070425231030@der-ingo.de> References: <644610523.20070330115430@der-ingo.de> <1286330289.20070425231030@der-ingo.de> Message-ID: <466E0BEF.4060000@etjohnson.us> Ingo Schmidt wrote: > Hi! > > I have finally had the time to finish documenting my conversion script > that I created and used to convert the VSS database at my company. > > You can find the script here: > > http://www.der-ingo.de/bin/vss2svn/convert.zip > > The included readme.txt is hopefully detailed enough to help everyone > get it going. > > I want to point out, that I am no expert in writing bash scripts, so > the script may not be perfect. > > However, it did get the job done very well. Any feedback is of course > welcome. > > vss2svn guys: > If you think this script is useful, feel free to put it into the > contribution section. > Thanks Ingo; I have added this under "autoconvert" in the contrib section. toby From finnied at aciworldwide.com Mon Jun 11 23:19:58 2007 From: finnied at aciworldwide.com (finnied@aciworldwide.com) Date: Mon Jun 11 23:20:04 2007 Subject: svnadmin: File not found In-Reply-To: <466E0A8F.2020404@etjohnson.us> Message-ID: vss2svn-users-bounces@lists.pumacode.org wrote on 12/06/2007 12:53:03: > finnied@aciworldwide.com wrote: > > > > Dirk & All, > > > > I came up with, and coded a solution for this problem. The patch is > > supplied below, if anyone is interested. > > > > To summarise, this is to fix the "svnadmin load" problem where it > > emits the following error: > > > > vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv > > > > > > svnadmin: File not found: transaction '20678-1', path > > 'orphaned/_YEDAAAAA/S01/InstallShield/WebGate/Setup Files/Uncompressed > > Files/Language Independent/OS Independent/setup.bmp' > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > > > The main problem stemmed from the following query in the > > BUILDACTIONHIST phase: > > > > my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' > > . 'itemtype ASC, priority ASC, sortkey ASC, action_id ASC'; > > > > If 2 or more projects were added at the exact same time, then > > depending on their physical VSS name, they may be returned by this > > query in the wrong order (i.e. ADDing a subproject before its parent > > project has been ADDed). > > > > To fix this, I decided to extend the use of "parentdata" in the > > PhysicalAction table, and instead treat it like a project "depth" > > attribute. During the MERGEPARENTDATA phase, all of the parent records > > are updated with their project path depth as an integer, such that the > > depth increases the further a project/file gets from the root project. > > > > The query is then changed to: > > > > my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' > > . 'itemtype ASC, priority ASC, parentdata ASC, sortkey > > ASC, action_id ASC'; > > > > The solution ensures that subprojects will have a parentdata value > > greater than that of their parent project, and therefore be returned > > by this query after their parent project. > > > > This fixed the problem for me. If anyone can think of a better > > solution, go for it. Delving into this problem has made me truly > > realise how badly designed VSS is ! > > > > A disclaimer - PERL is not my forte, so if there is a cleverer way of > > doing it, I'd be happy for people to edit my efforts :-) > > > > Dave > > > > Hi Dave, sorry it took so long to get back to you... as with the > previous patch, I don't have time to test this (and it's not an error I > personally ran into) but it looks logical at first glance so I went > ahead and committed in r313; if it breaks anyone else's conversion > hopefully we'll hear about it! > > However, I notice that you are not doing anything with the return value > of &GetPathDepth() on line 546... it seems maybe that you at first used > a return value then later decided to store it in the database instead? Yes, that's right. You could lose the assignment to $depth. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070612/f3e69275/attachment-0001.html From toby at etjohnson.us Tue Jun 12 00:19:33 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jun 12 00:19:35 2007 Subject: svnadmin: File not found In-Reply-To: References: Message-ID: <466E1ED5.6010708@etjohnson.us> finnied@aciworldwide.com wrote: > > vss2svn-users-bounces@lists.pumacode.org wrote on 12/06/2007 12:53:03: > > > finnied@aciworldwide.com wrote: > > > > > > Dirk & All, > > > > > > I came up with, and coded a solution for this problem. The patch is > > > supplied below, if anyone is interested. > > > > > > To summarise, this is to fix the "svnadmin load" problem where it > > > emits the following error: > > > > > > > vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv > > > > > > > > > > svnadmin: File not found: transaction '20678-1', path > > > 'orphaned/_YEDAAAAA/S01/InstallShield/WebGate/Setup > Files/Uncompressed > > > Files/Language Independent/OS Independent/setup.bmp' > > > > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > > > > > > > The main problem stemmed from the following query in the > > > BUILDACTIONHIST phase: > > > > > > my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' > > > . 'itemtype ASC, priority ASC, sortkey ASC, action_id > ASC'; > > > > > > If 2 or more projects were added at the exact same time, then > > > depending on their physical VSS name, they may be returned by this > > > query in the wrong order (i.e. ADDing a subproject before its parent > > > project has been ADDed). > > > > > > To fix this, I decided to extend the use of "parentdata" in the > > > PhysicalAction table, and instead treat it like a project "depth" > > > attribute. During the MERGEPARENTDATA phase, all of the parent > records > > > are updated with their project path depth as an integer, such that > the > > > depth increases the further a project/file gets from the root project. > > > > > > The query is then changed to: > > > > > > my $sql = 'SELECT * FROM PhysicalAction ORDER BY timestamp ASC, ' > > > . 'itemtype ASC, priority ASC, parentdata ASC, sortkey > > > ASC, action_id ASC'; > > > > > > The solution ensures that subprojects will have a parentdata value > > > greater than that of their parent project, and therefore be returned > > > by this query after their parent project. > > > > > > This fixed the problem for me. If anyone can think of a better > > > solution, go for it. Delving into this problem has made me truly > > > realise how badly designed VSS is ! > > > > > > A disclaimer - PERL is not my forte, so if there is a cleverer way of > > > doing it, I'd be happy for people to edit my efforts :-) > > > > > > Dave > > > > > > > Hi Dave, sorry it took so long to get back to you... as with the > > previous patch, I don't have time to test this (and it's not an error I > > personally ran into) but it looks logical at first glance so I went > > ahead and committed in r313; if it breaks anyone else's conversion > > hopefully we'll hear about it! > > > > However, I notice that you are not doing anything with the return value > > of &GetPathDepth() on line 546... it seems maybe that you at first used > > a return value then later decided to store it in the database instead? > > Yes, that's right. You could lose the assignment to $depth. OK, just wanted to make sure before I went and deleted it. Thanks again for the patch... toby From Aaron.Larson at honeywell.com Tue Jun 12 10:11:47 2007 From: Aaron.Larson at honeywell.com (Larson, Aaron (SWCOE)) Date: Tue Jun 12 10:11:56 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors In-Reply-To: <466E0746.20005@etjohnson.us> References: <466E0746.20005@etjohnson.us> Message-ID: >>>>> "TJ" == Toby writes: TJ> Erik Bray wrote: TJ> ... If anyone has problems with the resulting nightly build (or TJ> confirmation that it works correctly), let me know... Once a nightly comes out, I will test it out. I'm interested to see if the latest "file not found" error will resolve the situation I've been having. From toby at etjohnson.us Tue Jun 12 17:00:18 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jun 12 17:00:28 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors In-Reply-To: References: <466E0746.20005@etjohnson.us> Message-ID: <466F0962.3060203@etjohnson.us> Larson, Aaron (SWCOE) wrote: >>>>>> "TJ" == Toby writes: >>>>>> > > TJ> Erik Bray wrote: > TJ> ... If anyone has problems with the resulting nightly build (or > TJ> confirmation that it works correctly), let me know... > > Once a nightly comes out, I will test it out. I'm interested to see > if the latest "file not found" error will resolve the situation I've > been having. > A 12-Jun nightly build is already in the download directory: http://www.pumacode.org/download/vss2svn/nightly/ toby From Aaron.Larson at honeywell.com Wed Jun 13 08:48:50 2007 From: Aaron.Larson at honeywell.com (Larson, Aaron (SWCOE)) Date: Wed Jun 13 08:48:59 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors In-Reply-To: <466F0962.3060203@etjohnson.us> References: <466E0746.20005@etjohnson.us> <466F0962.3060203@etjohnson.us> Message-ID: >>>>>>> "TJ" == Toby writes: TJ> ... If anyone has problems with the resulting nightly build (or TJ> confirmation that it works correctly), let me know... Using the 070612-nightly build, the vss2svn run completed and generated a dumpfile. However, when I loaded the dumpfile into svn, it almost immediately died with: svnadmin: File not found: revision 50, path '/tools/installs/assemblers/borland' Previously the load would fail at roughly revision 4K. My SVN database is pretty screwed, so perhaps this should have died all along, but it hasn't in the past, at least not so early. Looking at the vss2svn log file the differences I see are: 1. 171 instances of: Use of uninitialized value in addition (+) at /PerlApp/Vss2Svn/Dumpfile.pm line 777. 2. The 070513-nightly had many more "Parent path missing while trying to add item '/orphaned...' at revision nnn: adding missing parents" messages. The 070612-nightly did have some, and at least one mentioned a similar path to the one that eventually caused the svnadmin load to fail: Attempt to commit to non-existant file '/tools/installs/assemblers/win32/borland/' at revision 50, changing to add; possibly missing recover 3. Where the 070513-nightly would get "could not open export file" error messages, the 070612-nightly seems to get one of the following instead (I didn't fully correlate them, there are ~150 of them): Parent path missing while trying to add item '...' at revision 17849: adding missing parents Attempt to commit to non-existant file '...' at revision 17849, changing to add; possibly missing recover Attempt to delete non-existent item '...' at revision 17844: possibly missing recover/add/share; skipping Now it is possible that the reason the above failed is due to my hacking on the VSS database. I've tried numerous things over the last couple of months, so perhaps I should start again with a reasonably fress vss database and try again. Also, I do have a database that I was previously successful in translating, so if you'd like I can try that one to see if the 070612-nightly still works on that. TJ> A 12-Jun nightly build is already in the download directory: TJ> http://www.pumacode.org/download/vss2svn/nightly/ Thanks for the nightly builds. They are very much appreciated. From toby at etjohnson.us Wed Jun 13 09:25:21 2007 From: toby at etjohnson.us (Toby Johnson) Date: Wed Jun 13 09:25:24 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors In-Reply-To: References: <466E0746.20005@etjohnson.us> <466F0962.3060203@etjohnson.us> Message-ID: <466FF041.2000702@etjohnson.us> Larson, Aaron (SWCOE) wrote: >>>>>>>> "TJ" == Toby writes: >>>>>>>> > > TJ> ... If anyone has problems with the resulting nightly build (or > TJ> confirmation that it works correctly), let me know... > > Using the 070612-nightly build, the vss2svn run completed and > generated a dumpfile. However, when I loaded the dumpfile into svn, > it almost immediately died with: > > svnadmin: File not found: revision 50, path > '/tools/installs/assemblers/borland' > > Previously the load would fail at roughly revision 4K. My SVN > database is pretty screwed, so perhaps this should have died all > along, but it hasn't in the past, at least not so early. > > Looking at the vss2svn log file the differences I see are: > > 1. 171 instances of: > Use of uninitialized value in addition (+) at > /PerlApp/Vss2Svn/Dumpfile.pm line 777. > > 2. The 070513-nightly had many more "Parent path missing while trying > to add item '/orphaned...' at revision nnn: adding missing > parents" messages. The 070612-nightly did have some, and at least > one mentioned a similar path to the one that eventually caused the > svnadmin load to fail: > > Attempt to commit to non-existant file > '/tools/installs/assemblers/win32/borland/' at revision 50, changing to > add; possibly missing recover > > 3. Where the 070513-nightly would get "could not open export file" > error messages, the 070612-nightly seems to get one of the > following instead (I didn't fully correlate them, there are ~150 > of them): > > Parent path missing while trying to add item '...' at revision > 17849: adding missing parents > Attempt to commit to non-existant file '...' at revision 17849, > changing to add; possibly missing recover > Attempt to delete non-existent item '...' at revision 17844: > possibly missing recover/add/share; skipping > > Now it is possible that the reason the above failed is due to my > hacking on the VSS database. I've tried numerous things over the last > couple of months, so perhaps I should start again with a reasonably > fress vss database and try again. Also, I do have a database that I > was previously successful in translating, so if you'd like I can try > that one to see if the 070612-nightly still works on that. > > TJ> A 12-Jun nightly build is already in the download directory: > TJ> http://www.pumacode.org/download/vss2svn/nightly/ > > Thanks for the nightly builds. They are very much appreciated. > Thanks for the testing and reporting back your findings. It is difficult to tell whether errors are due to corrupted VSS files or a bug in the script without further digging (however those "uninitialized value" warnings are coming from the output_content function just added so there is apparently some edge case there that is an issue). Let me know if you get a chance to try it on a "cleaner" database. toby From Aaron.Larson at honeywell.com Wed Jun 13 17:05:29 2007 From: Aaron.Larson at honeywell.com (Larson, Aaron (SWCOE)) Date: Wed Jun 13 17:05:49 2007 Subject: [PATCH] Use File::Copy to avoid out of memory errors In-Reply-To: <466FF041.2000702@etjohnson.us> References: <466E0746.20005@etjohnson.us> <466F0962.3060203@etjohnson.us> <466FF041.2000702@etjohnson.us> Message-ID: >>>>> "TJ" == Toby writes: AL> Larson, Aaron (SWCOE) wrote: TJ> ... If anyone has problems with the resulting nightly build (or TJ> confirmation that it works correctly), let me know... AL> Using the 070612-nightly build, the vss2svn run completed and AL> generated a dumpfile. However, when I loaded the dumpfile into svn, AL> it almost immediately died with: ... TJ> Let me know if you get a chance to try it on a "cleaner" database. I ran the script on a VSS database that I had previously been able to convert. The conversion succeeded. I looked at the resulting svnadmin load log and overall it looks pretty much the same. The order of commits within a single transaction is slightly different, so that causes some spurious differences that make doing a detailed analysis very timeconsuming, so I just spot checked, and all the ones I checked match. The resulting vss2svn log files are very similar. Some notable new messages: TASK: IMPORTSVN Use of uninitialized value in array element at /PerlApp/Vss2Svn/Dumpfile.pm line 167. ... Use of uninitialized value in addition (+) at /PerlApp/Vss2Svn/Dumpfile.pm line 777. Similar to the previous attempt I made, there seems to be a correlation between previous nightly build messages of the form: ERROR -- Could not open export file './_vss2svn/vssdata/DV/DVCAAAAA.1' and new messages: ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" get -b -v2 --force-overwrite -ewindows-1252 "../deos-level-e/data/D/DVCAAAAA" ./_vss2svn/vssdata/DV/DVCAAAAA) at vss2svn.pl line 1250 Use of uninitialized value in addition (+) at /PerlApp/Vss2Svn/Dumpfile.pm line 777. ssphys: reverse delta: failed to read necessary amount of data from input file Try `ssphys --help` for more information Or sometimes: FAILED with non-zero exit status 255 (cmd: "ssphys" get -b -v22 --force-overwrite -ewindows-1252 "../deos-level-e/data/D/DVCAAAAA" ./_vss2svn/vssdata/DV/DVCAAAAA) at vss2svn.pl line 1250 Its hard to be certain though. The output is pretty noisy. I'd have to reconstruct the previous SVN database to verify content equivalence. From hyugaricdeau at gmail.com Mon Jun 18 16:53:36 2007 From: hyugaricdeau at gmail.com (Erik Bray) Date: Mon Jun 18 16:53:41 2007 Subject: svnload: File already exists, redux Message-ID: Hi all, This issue has already come up in the past in threads such as this one: http://thread.gmane.org/gmane.comp.version-control.subversion.vss2svn.user/1184 However, I'm starting a new thread as that one is pretty old, and I'm experiencing the problem for what I think are different reasons. I'm still working on getting this 18 or so GB VSS repository loaded in to SVN. I can get the repository mostly loaded, but several revisions get skipped. There could be several reasons each one of these revisions is skipped, but I thought I'd start by looking at the first one where a problem occurs: revision 613. Looking at the db, the VssAction for 613 is something like this: 18948||NAEAAAAA|5|COMMIT|/Production/System Documents/Monster Compile Environment Install Notes/MonsterCompileEnviron.doc ...snip... /Production/Subsystems/ISS/Components/Facilities Database/Documents/Monster Compile Environment/MonsterCompileEnviron.doc ...snip... /Production/Subsystems/EPSQ/Components/Print Utility/Documents/Monster Compile Environment/MonsterCompileEnviron.doc|2|1| The long and short of it is the file MonsterCompileEnviron.doc was shared to a bunch of different locations where it already existed. Upon checking the svn dump file, I find that every action in that revision was a 'change' except for /Production/Subsystems/ISS/Components/Facilities Database/Documents/Monster Compile Environment/MonsterCompileEnviron.doc, which is the file where svn load gives the error message: svnadmin: File already exists: filesystem 'dss/db', transaction '612-1', path 'Production/Subsystems/ISS/Components/Facilities Database/Documents/Monster Compile Environment/MonsterCompileEnviron.doc' In my error log from vss2svn I have this error message: ERROR -- Attempt to commit to non-existant file '/Production/Subsystems/ISS/Components/Facilities Database/Documents/Monster Compile Environment/MonsterCompileEnviron.doc' at revision 613, changing to add; possibly missing recover at /usr/local/bin/vss2svn.pl line 1030 So there you have it. It was changed from a 'change' to an 'add'. But I wonder why. I looked in the code, and this error occurs if Vss2Svn::Dumpfile::SanityChecker::exists() returns false for that path. But that path definitely does exist in the repository at revision 612. So I will have to do some digging to see if I can figure out why exists() is returning the wrong result. Erik From u23104331 at telia.com Tue Jun 19 04:55:50 2007 From: u23104331 at telia.com (Micke) Date: Tue Jun 19 04:55:53 2007 Subject: Some issues Message-ID: <46779A16.5070307@telia.com> Hi! I've just tried a conversion of our latest product. Went smooth and looks good. Some quirks though: * Some Adds gets an extra initial empty line in the log messages. This seems to break the construction of changesets. Not all Adds with log messages but quite a large percentage. Adds with empty log messages get treated OK and bundle up with Modifys into changesets. * To move folders around, I tried to push the dump through SvnDumpTool. Initially it complained about a missing UUID. Added that by hand. Then it complained about "Text-content-md5", which as far as I could tell was missing in the revision indicated. It still failed after a load/dump cycle. I had planned to merge my old private branches with the new dump using SvnDumpTool, but that will obviously have to wait. Has anyone else tried SvnDumpTool (http://svn.borg.ch/svndumptool/) on vss2svn output? * The dump is full with .NET's test Add/Delete pairs. Has someone written a hack to filter them out? ----------------------- Revision-number: 4 .... svn:log V 97 Temporary file created by Visual Studio .NET to detect Microsoft Visual SourceSafe capabilities. ----------------------- Node-path: trunk/xxxx/~sak2d7a1b8b01c543f8.vcproj Node-kind: file Node-action: add Prop-content-length: 10 Text-content-length: 0 Content-length: 10 PROPS-END ----------------------- Node-path: trunk/xxxx/~sak2d94555e01c543f8.vcproj Node-action: delete ----------------------- (also note the extra empty line before the log message) Keep up the good work! /Micke From vss2svn at nogga.de Tue Jun 19 17:50:34 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 19 17:51:35 2007 Subject: Some issues In-Reply-To: <46779A16.5070307@telia.com> References: <46779A16.5070307@telia.com> Message-ID: <46784FAA.8010505@nogga.de> Hello Micke, I have added a fix for your first point. For the second issue, I don't have any idea about the UUID and the MD5. Is doesn't seem to be really necessary? A quick look into the SvnDumpTool also shows, that at least the UUID is optional. If we put a UUID into the dumpfile, should it be a random generated one, or should the user specify it on the command line? Best regards dirk Micke schrieb: > Hi! > > I've just tried a conversion of our latest product. Went smooth and > looks good. Some quirks though: > > * Some Adds gets an extra initial empty line in the log messages. This > seems to break the construction of changesets. Not all Adds with log > messages but quite a large percentage. Adds with empty log messages get > treated OK and bundle up with Modifys into changesets. > > * To move folders around, I tried to push the dump through SvnDumpTool. > Initially it complained about a missing UUID. Added that by hand. Then > it complained about "Text-content-md5", which as far as I could tell was > missing in the revision indicated. It still failed after a load/dump cycle. > > I had planned to merge my old private branches with the new dump using > SvnDumpTool, but that will obviously have to wait. > > Has anyone else tried SvnDumpTool (http://svn.borg.ch/svndumptool/) on > vss2svn output? > > * The dump is full with .NET's test Add/Delete pairs. Has someone > written a hack to filter them out? > ----------------------- > Revision-number: 4 > .... > svn:log > V 97 > > Temporary file created by Visual Studio .NET to detect Microsoft Visual > SourceSafe capabilities. > ----------------------- > Node-path: trunk/xxxx/~sak2d7a1b8b01c543f8.vcproj > Node-kind: file > Node-action: add > Prop-content-length: 10 > Text-content-length: 0 > Content-length: 10 > > PROPS-END > ----------------------- > Node-path: trunk/xxxx/~sak2d94555e01c543f8.vcproj > Node-action: delete > ----------------------- > > (also note the extra empty line before the log message) > > Keep up the good work! > /Micke > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > > From vss2svn at nogga.de Tue Jun 19 17:59:21 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 19 18:00:20 2007 Subject: svnload: File already exists, redux In-Reply-To: References: Message-ID: <467851B9.4050504@nogga.de> Hello Erik, thanks for looking this up, but it is still very difficult to tell you the exact course of events. The SanityChecker tries to follow all svn activities in order to detect such problem prior to loading the dumpfile into the repository. So the reason, that it will change an 'change' into an 'add' is really, that something went wrong in the history of this file. This could be * renames * delete / recover activities * Parent path renames * or all kind of strange parent paths activities. Is it possible for you to follow the flow of events for the parent path back to the root? When I read your mail I initially thought about the problem, that VSS can renamed deleted files via a share. So if you undelete a file, you will get the new name. But I expected that this is correctly tracked in the SanityChecker. Dirk From vss2svn at nogga.de Tue Jun 19 18:15:28 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 19 18:16:27 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <465725DC.9000102@etjohnson.us> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> Message-ID: <46785580.6020902@nogga.de> Hello Aleksey, just a dump question. Isn't this solved in the last nightly build with the "out of memory" path? This patch will copy the files verbatim into the dumpfile. So there shouldn't be a conversion involved. Or is should the dumpfile have only LF terminators, even if the host encoding is CR/LF? Best regards Dirk Toby Johnson schrieb: > Aleksey Nogin wrote: >> If I use the vss2svn (latest HEAD version) with the --auto_props >> option and the auto-props file sets svn:eol-style=native one some of >> the files, the files would not be converted correctly. The problem >> here is that the dump file is _required_ to only have the LF line >> terminators on files marked with svn:eol-style=native (see >> http://svn.haxx.se/dev/archive-2006-07/0981.shtml ), but vss2svn does >> not make any attempt to do that... >> >> Should I file an issue report on this? >> >> Aleksey > > Thanks Aleksy; I have disabled ticket submission by non-committers due > to ticket spam but have filed your issue as ticket 51 > (http://www.pumacode.org/projects/vss2svn/ticket/51). If you end up > getting code written to handle this we would appreciate a patch! > > toby > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > From vss2svn at nogga.de Tue Jun 19 18:23:43 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 19 18:24:42 2007 Subject: Some Comments In-Reply-To: <465608C5.2010200@etjohnson.us> References: <465608C5.2010200@etjohnson.us> Message-ID: <4678576F.3070404@nogga.de> > Most people find that creating and restoring a backup prior to > conversion will actually make the process more difficult. This is due > to the way VSS creates the timestamps when a restore is performed; it > appears that the contents of a folder were created after the folder > itself so there is no good way of determining where those files should go. I would like to add here, that you could get a better result if you use the following trick for the Archive/Restore cycle: * trun back the time on a local computer to a date prior to the first checkin in your database * create an empty database * set the time back to the real time. * now run the Import of your archive on this created database. The point is, that after the above steps you will have an archive, that has a creation date prior to your archived files. In the conversion step this will be recognized and hopefully the projects will go directly to the correct places, instead to orphaned. An explanation for the "success" for going through the archive/restore cycle is, that the database will be cleaned and also renumbered. This can help to sort out all possible quirks. Dirk From u23104331 at telia.com Wed Jun 20 10:37:31 2007 From: u23104331 at telia.com (Micke) Date: Wed Jun 20 10:37:35 2007 Subject: Some issues In-Reply-To: <46784FAA.8010505@nogga.de> References: <46779A16.5070307@telia.com> <46784FAA.8010505@nogga.de> Message-ID: <46793BAB.1010405@telia.com> Hi, The fix appears to be working. The dump shrank with a few hundred revisions. Thanks! The UUID is (AFAIK) used by the clients to identify the repository on the server. The generation of a new dump should in most cases result in a new UUID (At least I think it is rare that an existing WC out in the field would be possible to relocate to the new dump when it has been loaded). The md5:s on the other hand seem more tricky. When I load/dump, they aren't generated. In essence it seems that if the checksums are missing, they'll stay missing. The SvnDumpTool has the option the re-generate the checksums, but it can't handle that they are missing from the start. Almost a Catch-22, sigh! Best regards, /Micke (Anyone with a .NET filter hack? I've got 710*2 revisions with dummy Adds and Deletes.) Dirk wrote: > Hello Micke, > > I have added a fix for your first point. For the second issue, I don't > have any idea about the UUID and the MD5. Is doesn't seem to be really > necessary? A quick look into the SvnDumpTool also shows, that at least > the UUID is optional. > > If we put a UUID into the dumpfile, should it be a random generated one, > or should the user specify it on the command line? > > Best regards > dirk > > Micke schrieb: >> Hi! >> >> I've just tried a conversion of our latest product. Went smooth and >> looks good. Some quirks though: >> >> * Some Adds gets an extra initial empty line in the log messages. This >> seems to break the construction of changesets. Not all Adds with log >> messages but quite a large percentage. Adds with empty log messages get >> treated OK and bundle up with Modifys into changesets. >> >> * To move folders around, I tried to push the dump through SvnDumpTool. >> Initially it complained about a missing UUID. Added that by hand. Then >> it complained about "Text-content-md5", which as far as I could tell was >> missing in the revision indicated. It still failed after a load/dump >> cycle. >> >> I had planned to merge my old private branches with the new dump using >> SvnDumpTool, but that will obviously have to wait. >> >> Has anyone else tried SvnDumpTool (http://svn.borg.ch/svndumptool/) on >> vss2svn output? >> >> * The dump is full with .NET's test Add/Delete pairs. Has someone >> written a hack to filter them out? >> ----------------------- >> Revision-number: 4 >> .... >> svn:log >> V 97 >> >> Temporary file created by Visual Studio .NET to detect Microsoft Visual >> SourceSafe capabilities. >> ----------------------- >> Node-path: trunk/xxxx/~sak2d7a1b8b01c543f8.vcproj >> Node-kind: file >> Node-action: add >> Prop-content-length: 10 >> Text-content-length: 0 >> Content-length: 10 >> >> PROPS-END >> ----------------------- >> Node-path: trunk/xxxx/~sak2d94555e01c543f8.vcproj >> Node-action: delete >> ----------------------- >> >> (also note the extra empty line before the log message) >> >> Keep up the good work! >> /Micke >> >> _______________________________________________ >> vss2svn-users mailing list >> Project homepage: >> http://www.pumacode.org/projects/vss2svn/ >> Subscribe/Unsubscribe/Admin: >> http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org >> >> Mailing list web interface (with searchable archives): >> http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user >> >> >> >> > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > From vss2svn at nogga.de Thu Jun 21 06:01:37 2007 From: vss2svn at nogga.de (Dirk) Date: Thu Jun 21 06:03:43 2007 Subject: Some issues In-Reply-To: <46793BAB.1010405@telia.com> References: <46779A16.5070307@telia.com> <46784FAA.8010505@nogga.de> <46793BAB.1010405@telia.com> Message-ID: <467A4C81.9070600@nogga.de> Hi, > The md5:s on the other hand seem more tricky. When I load/dump, they > aren't generated. In essence it seems that if the checksums are missing, > they'll stay missing. > The SvnDumpTool has the option the re-generate the checksums, but it > can't handle that they are missing from the start. Almost a Catch-22, sigh! > If it is a simply MD5 checksum, we could add this to the handler. I just don't know the perl module to use... Any ideas? > (Anyone with a .NET filter hack? I've got 710*2 revisions with dummy > Adds and Deletes.) > I think, this could be easily done via a new state "RemoveDotNetCheckin", e.g by cloning the MergeMoveData function. Then search for all "ADD" actions with a specific comment, get the phyiscal ID and remove all vss actions with this physical ID. Dirk From vss2svn at nogga.de Thu Jun 21 06:15:26 2007 From: vss2svn at nogga.de (Dirk) Date: Thu Jun 21 06:17:08 2007 Subject: svnload: File already exists, redux In-Reply-To: References: <467851B9.4050504@nogga.de> Message-ID: <467A4FBE.30308@nogga.de> Hi, > > For now I just modified the dump file by hand, but it's still curious. Nevertheless, it would be interesting to know the details. In many cases one problem is the door to a bunch of other problems ;-) > > Fixing that problem solved a number of other problems until I reached > revision 3800. This is an ugly one: > > At revision 3794 a label called DSS-Demo-12-02 is created from the > path /P3I/Software Engineering: > 31261||ANEDAAAA|4|LABEL|/P3I/Software Engineering/|1|0|DSS-Demo-12-02 > > That works out fine. However, the path '/P3I/Software Engineering' > has a subdirectory called 'System Component' which, in revision 3795 > gets renamed to 'SystemComponent' (the space is removed): > 31262|ANEDAAAA|CNEDAAAA||RENAME|/P3I/Software Engineering/System > Component/|1|0|SystemComponent/ > > Finally, in revision 3800 the path '/P3I/Software > Engineering/SystemComponent/ISI/Common/' is added to the same label: > 31771||JREDAAAA|5|LABEL|/P3I/Software > Engineering/SystemComponent/ISI/Common/|1|0|DSS-Demo-12-02 > > The path 'labels/DSS-Demo-12-02/P3I/Software Engineering/System > Component/ISI/Common' already exists from the previous LABEL action. > However 'labels/DSS-Demo-12-02/P3I/Software > Engineering/SystemComponent/ISI/Common' (without no space in > 'SystemComponent') does not exist, since it was LABELed before the > RENAME. > > The associated rows from PhysicalAction are: > 2868|ANEDAAAA|4||LABEL|/|1|1040401208|Robertj|0||5|AAAADENA|0|DSS-Demo-12-02|Version > > used for the December 18th demonstration to DSS > 2869|CNEDAAAA||ANEDAAAA|RENAME|System > Component/|1|1040634208|Robertj|0|SystemComponent/|5|AAAADENC|1|| > 63525|JREDAAAA|5||LABEL|/|1|1040636589|Robertj|0||5|AAAADERJ|0|DSS-Demo-12-02| > > > Clearly the timestamp for the RENAME is later than the first LABEL. > But I can't help but to feel the two should have happened in the > reverse order. Ah well, this one can probably just be chalked up to > weirdness in the repository. I'm not sure there's anything the script > could have done to catch something like this. Interesting finding. The problem is, that there is no "label path" concept in VSS. Labels are concrete versions or attached to other versions in projects or files. Label promotion is the problem here. You have created a label at a higher level of the project tree and later assigned the label to some other sub project. You expect, that if you checkout by label, you will get all files from the "parent project" at the time of the first labeling, and the files from the subproject from the second labeling. Now the big surprise: If you have renamed a file after the first labeling, you will get it with the new filename and not the filename, that was right at the time of the labeling. If this was a source code file, you are in trouble now, since you can't build your project anymore. :-((( Therfore renaming is a big "NoGo" in VSS. Similiar problem for renaming shares, Grrrrrr.... filenames and project names are not "versioned" in a way that subversion can do this. It would be a dramatic failure to reflect every rename into each label in order to deal with the situation, this would copy the problem from VSS to subversion, and this is one of my main reasons to convert to subversion. Since we have the unique phyiscal id, it should be possible to align the phyiscal ids during the second labeling, but what would you expect? A rename in the label path in order to get to the new name? A labeling activity with the old name? If the dumpfile is not loadable, then we should fix the "SanityChecker" to deal with this situation. Best regards Dirk From vss2svn at nogga.de Fri Jun 22 10:53:39 2007 From: vss2svn at nogga.de (Dirk) Date: Fri Jun 22 10:55:15 2007 Subject: Some issues In-Reply-To: <467A4C81.9070600@nogga.de> References: <46779A16.5070307@telia.com> <46784FAA.8010505@nogga.de> <46793BAB.1010405@telia.com> <467A4C81.9070600@nogga.de> Message-ID: <467BE273.3020908@nogga.de> Hi, >> (Anyone with a .NET filter hack? I've got 710*2 revisions with dummy >> Adds and Deletes.) >> > I think, this could be easily done via a new state > "RemoveDotNetCheckin", e.g by cloning the MergeMoveData function. Then > search for all "ADD" actions with a specific comment, get the phyiscal > ID and remove all vss actions with this physical ID. I have created a small patch, that is not really tested, it work for me, but I have only 7 dummy checkins. Nevertheless I found one project directory with the same comment, but the project name itself was a real name and not a temporary one. So I excluded projects from beeing deleted. Dirk From vss2svn at nogga.de Fri Jun 22 10:55:40 2007 From: vss2svn at nogga.de (Dirk) Date: Fri Jun 22 10:57:17 2007 Subject: Some issues In-Reply-To: <467BE273.3020908@nogga.de> References: <46779A16.5070307@telia.com> <46784FAA.8010505@nogga.de> <46793BAB.1010405@telia.com> <467A4C81.9070600@nogga.de> <467BE273.3020908@nogga.de> Message-ID: <467BE2EC.3060709@nogga.de> Dirk schrieb: > Hi, > >>> (Anyone with a .NET filter hack? I've got 710*2 revisions with dummy >>> Adds and Deletes.) >>> >> I think, this could be easily done via a new state >> "RemoveDotNetCheckin", e.g by cloning the MergeMoveData function. >> Then search for all "ADD" actions with a specific comment, get the >> phyiscal ID and remove all vss actions with this physical ID. > > I have created a small patch, that is not really tested, it work for > me, but I have only 7 dummy checkins. Nevertheless I found one project > directory with the same comment, but the project name itself was a > real name and not a temporary one. So I excluded projects from beeing > deleted. > Sorry, forgot to attach the patch. Dirk -------------- next part -------------- Index: vss2svn.pl =================================================================== --- vss2svn.pl (revision 315) +++ vss2svn.pl (working copy) @@ -70,8 +70,12 @@ # Merge data from move actions MERGEMOVEDATA => {handler => \&MergeMoveData, - next => 'BUILDACTIONHIST'}, + next => 'REMOVETMPCHECKIN'}, + # Remove temporary check ins + REMOVETMPCHECKIN => {handler => \&RemoveTemporaryCheckIns, + next => 'BUILDACTIONHIST'}, + # Take the history of physical actions and convert them to VSS # file actions BUILDACTIONHIST => {handler => \&BuildVssActionHistory, @@ -773,6 +777,42 @@ } # End MergeMoveData ############################################################################### +# RemoveTemporaryCheckIns +# remove temporary checkins that where create to detect MS VSS capabilities +############################################################################### +sub RemoveTemporaryCheckIns { + my($sth, $rows, $row); + $sth = $gCfg{dbh}->prepare('SELECT * FROM PhysicalAction ' + . 'WHERE comment = "Temporary file created by Visual Studio .NET to detect Microsoft Visual SourceSafe capabilities."' + . ' AND actiontype = "ADD"' + . ' AND itemtype = 2'); # only delete files, not projects + $sth->execute(); + + # need to pull in all recs at once, since we'll be updating/deleting data + $rows = $sth->fetchall_arrayref( {} ); + + foreach $row (@$rows) { + my $physname = $row->{physname}; + + my $sql = 'SELECT * FROM PhysicalAction WHERE physname = ?'; + my $update = $gCfg{dbh}->prepare($sql); + + $update->execute( $physname ); + + # need to pull in all recs at once, since we'll be updating/deleting data + my $recs = $update->fetchall_arrayref( {} ); + + foreach my $rec (@$recs) { + print "Remove action_id $rec->{action_id}, $rec->{physname}, $rec->{actiontype}, $rec->{itemname}\n"; + print " $rec->{comment}\n" if defined ($rec->{comment}); + &DeleteChildRec($rec->{action_id}); + } + } + + 1; +} + +############################################################################### # DeleteChildRec ############################################################################### sub DeleteChildRec { @@ -1867,7 +1907,8 @@ --resume : Resume a failed or aborted previous run --task : specify the task to resume; task is one of the following INIT, LOADVSSNAMES, FINDDBFILES, GETPHYSHIST, - MERGEPARENTDATA, BUILDACTIONHIST, IMPORTSVN + MERGEPARENTDATA, MERGEMOVEDATA, REMOVETMPCHECKIN, + BUILDACTIONHIST, IMPORTSVN --verbose : Print more info about the items being processed --debug : Print lots of debugging info. From u23104331 at telia.com Sun Jun 24 15:56:32 2007 From: u23104331 at telia.com (Micke) Date: Sun Jun 24 15:56:36 2007 Subject: Some issues In-Reply-To: <467BE2EC.3060709@nogga.de> References: <46779A16.5070307@telia.com> <46784FAA.8010505@nogga.de> <46793BAB.1010405@telia.com> <467A4C81.9070600@nogga.de> <467BE273.3020908@nogga.de> <467BE2EC.3060709@nogga.de> Message-ID: <467ECC70.2000402@telia.com> Dirk wrote: > Dirk schrieb: >> Hi, >> >>>> (Anyone with a .NET filter hack? I've got 710*2 revisions with dummy >>>> Adds and Deletes.) >>>> >>> I think, this could be easily done via a new state >>> "RemoveDotNetCheckin", e.g by cloning the MergeMoveData function. >>> Then search for all "ADD" actions with a specific comment, get the >>> phyiscal ID and remove all vss actions with this physical ID. >> >> I have created a small patch, that is not really tested, it work for >> me, but I have only 7 dummy checkins. Nevertheless I found one project >> directory with the same comment, but the project name itself was a >> real name and not a temporary one. So I excluded projects from beeing >> deleted. >> > Sorry, forgot to attach the patch. Sorry, upgraded to Fedora 7. I've been wrestling all kinds of dysfunctional stuff the last few days. "at" still ends of with error message "atd[29802]: Open of jobfile failed: Success". Sigh! Next week, I'll be busy, but I'll get back to you as soon as I can. /Micke From toby at etjohnson.us Sun Jun 24 16:13:18 2007 From: toby at etjohnson.us (Toby Johnson) Date: Sun Jun 24 16:13:21 2007 Subject: Some issues In-Reply-To: <467A4C81.9070600@nogga.de> References: <46779A16.5070307@telia.com> <46784FAA.8010505@nogga.de> <46793BAB.1010405@telia.com> <467A4C81.9070600@nogga.de> Message-ID: <467ED05E.7090106@etjohnson.us> Dirk wrote: > Hi, > >> The md5:s on the other hand seem more tricky. When I load/dump, they >> aren't generated. In essence it seems that if the checksums are missing, >> they'll stay missing. >> The SvnDumpTool has the option the re-generate the checksums, but it >> can't handle that they are missing from the start. Almost a Catch-22, >> sigh! >> > If it is a simply MD5 checksum, we could add this to the handler. I > just don't know the perl module to use... Any ideas? Looks like Digest::MD5 would be the way to go: http://search.cpan.org/~gaas/Digest-MD5-2.36/MD5.pm toby From toby at etjohnson.us Sun Jun 24 16:22:55 2007 From: toby at etjohnson.us (Toby Johnson) Date: Sun Jun 24 16:22:58 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <46785580.6020902@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> Message-ID: <467ED29F.70308@etjohnson.us> Dirk wrote: > Hello Aleksey, > > just a dump question. Isn't this solved in the last nightly build with > the "out of memory" path? This patch will copy the files verbatim into > the dumpfile. So there shouldn't be a conversion involved. Or is > should the dumpfile have only LF terminators, even if the host > encoding is CR/LF? > > Best regards > Dirk > > Toby Johnson schrieb: >> Aleksey Nogin wrote: >>> If I use the vss2svn (latest HEAD version) with the --auto_props >>> option and the auto-props file sets svn:eol-style=native one some of >>> the files, the files would not be converted correctly. The problem >>> here is that the dump file is _required_ to only have the LF line >>> terminators on files marked with svn:eol-style=native (see >>> http://svn.haxx.se/dev/archive-2006-07/0981.shtml ), but vss2svn >>> does not make any attempt to do that... >>> >>> Should I file an issue report on this? >>> >>> Aleksey (Replying to Dirk's reply above...) I don't think that the problem is "solved" by the out-of-memory patch. The problem as I understand it is that any file which has the svn:eol-style=native is *required* to have only LF terminators in the dumpfile. Of course most source files in the typical VSS repo will have CR+LF terminators, so regardless of how we actually copy the contents into the dumpfile, it will need to be converted in order to produce a valid dumpfile. I wonder if File::Copy allows some sort of "hook" so that we can process each "chunk" on the fly to do this conversion? Otherwise it may be necessary to create an intermediate file in order to perform the conversion prior to the final copy to the dumpfile... toby From vss2svn at nogga.de Mon Jun 25 03:59:23 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jun 25 04:01:23 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <467ED29F.70308@etjohnson.us> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> Message-ID: <467F75DB.7060408@nogga.de> > I wonder if File::Copy allows some sort of "hook" so that we can > process each "chunk" on the fly to do this conversion? Otherwise it > may be necessary to create an intermediate file in order to perform > the conversion prior to the final copy to the dumpfile... Hello Toby, this is where my PERL knowledge completely stops. At this point I would start an external converter to do the job ;-). As far as I have seen this copy patch, but wouldn't be a block read and block write perform a similar functionality? Somewehere I read, that you can set filters on streams in PERL. So wouldn't be a chuncked implementation of the copy with a CR+LF input stream and a LF output stream also do the job? Dirk From vss2svn at nogga.de Mon Jun 25 05:28:52 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jun 25 05:30:31 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <467F75DB.7060408@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> Message-ID: <467F8AD4.2020509@nogga.de> Dirk schrieb: > >> I wonder if File::Copy allows some sort of "hook" so that we can >> process each "chunk" on the fly to do this conversion? Otherwise it >> may be necessary to create an intermediate file in order to perform >> the conversion prior to the final copy to the dumpfile... > Hello Toby, > > this is where my PERL knowledge completely stops. At this point I > would start an external converter to do the job ;-). As far as I have > seen this copy patch, but wouldn't be a block read and block write > perform a similar functionality? Somewehere I read, that you can set > filters on streams in PERL. So wouldn't be a chuncked implementation > of the copy with a CR+LF input stream and a LF output stream also do > the job? A quick galnce into the doucmentation: http://perldoc.perl.org/File/Copy.html *Note that passing in files as handles instead of names may lead to loss of information on some operating systems; it is recommended that you use file names whenever possible.* Files are opened in binary mode where applicable. To get a consistent behaviour when copying from a filehandle to a file, use |binmode | on the filehandle. So wouldn't something like: if(!defined $text && defined $file) { if ($node->get_prop('svn:mime-type') == 'application/octet-stream'); open ($input, "<:crlf", $file) } else { open ($input, "<", $file); binmode ($input); } copy($input, $fh); print $fh "\n"; } Dirk From vss2svn at nogga.de Mon Jun 25 12:22:55 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jun 25 12:24:30 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <467F8AD4.2020509@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> Message-ID: <467FEBDF.1030704@nogga.de> Grrr. this doesn't work. I tried different methods, but the :crlf has no influence. Perhaps I'm doing something stupidly wrong. Simply converting in several chunks is also not possible, since we could split up in between a CR and LF. I think the only solution is to run an external utility. But which one? I only know of dos2unix, but this is not installed on a windows system. I also tried to call perl -p -e 's/\r\n/\n/g' /inputfile/ > /outputfile/ but interestingly this doesn't work either. Perhaps I'm doing something really stupid here. Dirk > > So wouldn't something like: > > if(!defined $text && defined $file) { > if ($node->get_prop('svn:mime-type') == > 'application/octet-stream'); > open ($input, "<:crlf", $file) > } else { > open ($input, "<", $file); > binmode ($input); > } > copy($input, $fh); > print $fh "\n"; > } > > > > Dirk > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > From toby at etjohnson.us Mon Jun 25 20:46:54 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jun 25 20:46:57 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <467F8AD4.2020509@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> Message-ID: <468061FE.10905@etjohnson.us> Dirk wrote: > Dirk schrieb: >> >>> I wonder if File::Copy allows some sort of "hook" so that we can >>> process each "chunk" on the fly to do this conversion? Otherwise it >>> may be necessary to create an intermediate file in order to perform >>> the conversion prior to the final copy to the dumpfile... >> Hello Toby, >> >> this is where my PERL knowledge completely stops. At this point I >> would start an external converter to do the job ;-). As far as I have >> seen this copy patch, but wouldn't be a block read and block write >> perform a similar functionality? Somewehere I read, that you can set >> filters on streams in PERL. So wouldn't be a chuncked implementation >> of the copy with a CR+LF input stream and a LF output stream also do >> the job? > A quick galnce into the doucmentation: > http://perldoc.perl.org/File/Copy.html > > *Note that passing in files as handles instead of names may lead to > loss of information on some operating systems; it is recommended that > you use file names whenever possible.* Files are opened in binary mode > where applicable. To get a consistent behaviour when copying from a > filehandle to a file, use |binmode > | on the filehandle. > > So wouldn't something like: > > if(!defined $text && defined $file) { > if ($node->get_prop('svn:mime-type') == > 'application/octet-stream'); > open ($input, "<:crlf", $file) > } else { > open ($input, "<", $file); > binmode ($input); > } > copy($input, $fh); > print $fh "\n"; > } Wow, I had never even heard of these filehandle "layers". See, you have taught me some new Perl tricks :) After doing some more experimentation I have found that the File::Copy does not seem to respect these layers at all, but that the following code works on both Windows and *nix to do the translation while copying from one filehandle to the other: if(!defined $text && defined $file) { if ($node->get_prop('svn:mime-type') == 'application/octet-stream'); open ($input, "<:crlf", $file); while(<$input>) { print $fh $_; } } else { open ($input, "<", $file); binmode ($input); copy($input, $fh); } close $input; print $fh "\n"; } I'm sure that the "while(<$input>)" is probably not as efficient as File::Copy, but it will at least be buffered by the Perl I/O routines and at any rate it is still reading only one line at a time, so it will not suffer the memory issues of reading in the whole contents as we did before. toby From toby at etjohnson.us Mon Jun 25 20:48:40 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jun 25 20:48:43 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <467FEBDF.1030704@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <467FEBDF.1030704@nogga.de> Message-ID: <46806268.9080206@etjohnson.us> Dirk wrote: > Grrr. this doesn't work. I tried different methods, but the :crlf has > no influence. Perhaps I'm doing something stupidly wrong. Simply > converting in several chunks is also not possible, since we could > split up in between a CR and LF. I think the only solution is to run > an external utility. But which one? I only know of dos2unix, but this > is not installed on a windows system. I also tried to call > > perl -p -e 's/\r\n/\n/g' /inputfile/ > /outputfile/ > > but interestingly this doesn't work either. Perhaps I'm doing > something really stupid here. Nothing stupid, your code snippet above will work correctly on *nix. The problem is that on Windows, it treats "\n" as "\r\n" by default (unless binmode is specified), so the regex will never match. toby From thomas.franconville at sysdeo.fr Tue Jun 26 03:37:39 2007 From: thomas.franconville at sysdeo.fr (Thomas Franconville) Date: Tue Jun 26 04:09:30 2007 Subject: ssphys: unknown action 21 at offset 0x1a0 Message-ID: <1182843459.4680c243d12bd@webmail.sysdeo.fr> Hello, I have problems with converting a vss project with the nightly build version of vss2svn D:\Program\vss2svn>vss2svn.exe --verbose --vssdir D:\VSS\DELIVERY-3.3 Adding 'C:/DOCUME~1/TFRANC~1/LOCALS~1/Temp/pdk-tfranconville-2356/encodings' to encodings file path Connecting to database ./_vss2svn/vss_data.db ======== VSS2SVN ======== BEGINNING CONVERSION... Start Time : Mon Jun 25 15:45:00 2007 VSS Dir : D:\VSS\DELIVERY-3.3SETTING STEP 0 TASK: GETPHYSHIST STARTING CACHE FOR PhysicalAction "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" ssphys: unknown action 21 at offset 0x1a0 Try `ssphys --help` for more information ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") at vss2svn.pl line 1251 Temp Dir : ./_vss2svn Dumpfile : vss2svn-dumpfile.dat VSS Encoding : windows-1252 VSS2SVN ver : 0.11.0-nightly.315 SSPHYS exe : ssphys SSPHYS ver : 0.22.0.275 XML Parser : XML::SAX::Expat TASK: INIT SETTING TASK LOADVSSNAMES SETTING STEP 0 TASK: LOADVSSNAMES "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/names.dat" STARTING CACHE FOR NameLookup COMMITTING CACHE 'NameLookup' TO DATABASE SETTING TASK FINDDBFILES SETTING STEP 0 TASK: FINDDBFILES STARTING CACHE FOR Physical COMMITTING CACHE 'Physical' TO DATABASE SETTING TASK GETPHYSHIST SETTING STEP 0 TASK: GETPHYSHIST STARTING CACHE FOR PhysicalAction "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" ssphys: unknown action 21 at offset 0x1a0 Try `ssphys --help` for more information ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") at vss2svn.pl line 1251 I have a lot of time this kind of error (ssphys: unknown action 21 at offset 0x1a0) And my execution finish like that: WARNING: control character 0x16 in text input at character 206 not well-formed (invalid token) at line 38, column 106, byte 1552 at /PerlApp/XML/Parser.pm line 187 With the version 0.11.0 alpha 1 I have a lot of this kind of warning : WARNING: Unknown action 'ArchiveVersionsofFile' "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQDAAAAA" WARNING: Unknown action 'ArchiveVersionsofFile' "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQEAAAAA" ssphys: unknown action 21 at offset 0x1a0 Try `ssphys --help` for more information ERROR -- FAILED with non-zero exit status 255 at vss2svn.pl line 1092 "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQFAAAAA" Then it finish the execution but lost a lot of files directories. Like this: at vss2svn.pl line 851 Parent path missing while trying to add item '/orphaned/_DOIAAAAA/deliverycursor/' at revision 42: adding missing parents Who can help me ? I have done vssbinPath/ANALYZE.EXE -banalyze-results-fix -f -d -v4 -i- data and there are no more errors. Thank?s Thomas ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From vss2svn at nogga.de Tue Jun 26 04:23:37 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 26 04:25:11 2007 Subject: ssphys: unknown action 21 at offset 0x1a0 In-Reply-To: <1182843459.4680c243d12bd@webmail.sysdeo.fr> References: <1182843459.4680c243d12bd@webmail.sysdeo.fr> Message-ID: <4680CD09.6010609@nogga.de> Hello Thomas, this means, that you used some type of vss activity, that is not decoded yet. In other words, you did something, that nobody else who used the converter did up to today ;-) If your archive is still intact, can you please tell me the list of activities on your root project? There must be something unusual in this history. Not common tasks, like create project, create file, rename delete and so on. Dirk Thomas Franconville schrieb: > Hello, > > > > I have problems with converting a vss project with the nightly build version of > vss2svn > > > > > > D:\Program\vss2svn>vss2svn.exe --verbose --vssdir D:\VSS\DELIVERY-3.3 > > Adding 'C:/DOCUME~1/TFRANC~1/LOCALS~1/Temp/pdk-tfranconville-2356/encodings' to > encodings file path > > Connecting to database ./_vss2svn/vss_data.db > > > > ======== VSS2SVN ======== > > BEGINNING CONVERSION... > > Start Time : Mon Jun 25 15:45:00 2007 > > > > > > VSS Dir : D:\VSS\DELIVERY-3.3SETTING STEP 0 > > TASK: GETPHYSHIST > > > > STARTING CACHE FOR PhysicalAction > > "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > > ssphys: unknown action 21 at offset 0x1a0 > > Try `ssphys --help` for more information > > ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 > "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > > at vss2svn.pl line 1251 > > Temp Dir : ./_vss2svn > > Dumpfile : vss2svn-dumpfile.dat > > VSS Encoding : windows-1252 > > > > VSS2SVN ver : 0.11.0-nightly.315 > > SSPHYS exe : ssphys > > SSPHYS ver : 0.22.0.275 > > XML Parser : XML::SAX::Expat > > > > TASK: INIT > > > > SETTING TASK LOADVSSNAMES > > > > SETTING STEP 0 > > TASK: LOADVSSNAMES > > "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/names.dat" > > > > STARTING CACHE FOR NameLookup > > > > > > COMMITTING CACHE 'NameLookup' TO DATABASE > > > > SETTING TASK FINDDBFILES > > > > SETTING STEP 0 > > TASK: FINDDBFILES > > > > STARTING CACHE FOR Physical > > > > > > COMMITTING CACHE 'Physical' TO DATABASE > > > > SETTING TASK GETPHYSHIST > > > > SETTING STEP 0 > > TASK: GETPHYSHIST > > > > STARTING CACHE FOR PhysicalAction > > "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > > ssphys: unknown action 21 at offset 0x1a0 > > Try `ssphys --help` for more information > > ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 > "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > > at vss2svn.pl line 1251 > > > > > > I have a lot of time this kind of error (ssphys: unknown action 21 at offset > 0x1a0) > > > > And my execution finish like that: > > > > WARNING: control character 0x16 in text input at character 206 > > > > not well-formed (invalid token) at line 38, column 106, byte 1552 at > /PerlApp/XML/Parser.pm line 187 > > > > > > > > > > > > > > With the version 0.11.0 alpha 1 I have a lot of this kind of warning : > > > > WARNING: Unknown action 'ArchiveVersionsofFile' > > "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQDAAAAA" > > > > WARNING: Unknown action 'ArchiveVersionsofFile' > > "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQEAAAAA" > > ssphys: unknown action 21 at offset 0x1a0 > > Try `ssphys --help` for more information > > ERROR -- FAILED with non-zero exit status 255 > > at vss2svn.pl line 1092 > > "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQFAAAAA" > > > > Then it finish the execution but lost a lot of files directories. > > > > Like this: > > > > at vss2svn.pl line 851 > > Parent path missing while trying to add item > '/orphaned/_DOIAAAAA/deliverycursor/' at revision 42: adding missing parents > > > > > > > > Who can help me ? > > > > I have done vssbinPath/ANALYZE.EXE -banalyze-results-fix -f -d -v4 -i- data > and there are no more errors. > > Thank?s > > Thomas > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > > From vss2svn at nogga.de Tue Jun 26 04:49:19 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 26 04:51:26 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <468061FE.10905@etjohnson.us> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> Message-ID: <4680D30F.4050003@nogga.de> Hi, > Wow, I had never even heard of these filehandle "layers". See, you > have taught me some new Perl tricks :) You're welcome ;-) > if(!defined $text && defined $file) { > if ($node->get_prop('svn:mime-type') == > 'application/octet-stream'); actually this condition should read my $style = $props->{'svn:eol-style'}; if (defined $style && $style eq 'native') { I tried your suggestion and from what I can see it looks good. But I don't use a mixed setup, so someone else should try this out. I will commit my changes later this day, so that the next nighly build will have the modification. Dirk From thomas.franconville at sysdeo.fr Tue Jun 26 04:37:13 2007 From: thomas.franconville at sysdeo.fr (Thomas Franconville) Date: Tue Jun 26 05:09:03 2007 Subject: ssphys: unknown action 21 at offset 0x1a0 In-Reply-To: <4680CD09.6010609@nogga.de> References: <1182843459.4680c243d12bd@webmail.sysdeo.fr> <4680CD09.6010609@nogga.de> Message-ID: <1182847033.4680d0397c476@webmail.sysdeo.fr> Hello Dirk Thank's for your help. Here are the different activities found with the history of the administration console (translated from french): - Commit - Add file / project - Delete file / project - Label - Destroy file / project - Rename file / project There is nothing really special :( Is that what you want, or is there is a specific operation/file to have the list of activities ? Cheers Thomas Selon Dirk : > Hello Thomas, > > this means, that you used some type of vss activity, that is not decoded > yet. In other words, you did something, that nobody else who used the > converter did up to today ;-) > > If your archive is still intact, can you please tell me the list of > activities on your root project? There must be something unusual in this > history. Not common tasks, like create project, create file, rename > delete and so on. > > > Dirk > > > Thomas Franconville schrieb: > > Hello, > > > > > > > > I have problems with converting a vss project with the nightly build > version of > > vss2svn > > > > > > > > > > > > D:\Program\vss2svn>vss2svn.exe --verbose --vssdir D:\VSS\DELIVERY-3.3 > > > > Adding > 'C:/DOCUME~1/TFRANC~1/LOCALS~1/Temp/pdk-tfranconville-2356/encodings' to > > encodings file path > > > > Connecting to database ./_vss2svn/vss_data.db > > > > > > > > ======== VSS2SVN ======== > > > > BEGINNING CONVERSION... > > > > Start Time : Mon Jun 25 15:45:00 2007 > > > > > > > > > > > > VSS Dir : D:\VSS\DELIVERY-3.3SETTING STEP 0 > > > > TASK: GETPHYSHIST > > > > > > > > STARTING CACHE FOR PhysicalAction > > > > "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > > > > ssphys: unknown action 21 at offset 0x1a0 > > > > Try `ssphys --help` for more information > > > > ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info > -ewindows-1252 > > "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > > > > at vss2svn.pl line 1251 > > > > Temp Dir : ./_vss2svn > > > > Dumpfile : vss2svn-dumpfile.dat > > > > VSS Encoding : windows-1252 > > > > > > > > VSS2SVN ver : 0.11.0-nightly.315 > > > > SSPHYS exe : ssphys > > > > SSPHYS ver : 0.22.0.275 > > > > XML Parser : XML::SAX::Expat > > > > > > > > TASK: INIT > > > > > > > > SETTING TASK LOADVSSNAMES > > > > > > > > SETTING STEP 0 > > > > TASK: LOADVSSNAMES > > > > "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/names.dat" > > > > > > > > STARTING CACHE FOR NameLookup > > > > > > > > > > > > COMMITTING CACHE 'NameLookup' TO DATABASE > > > > > > > > SETTING TASK FINDDBFILES > > > > > > > > SETTING STEP 0 > > > > TASK: FINDDBFILES > > > > > > > > STARTING CACHE FOR Physical > > > > > > > > > > > > COMMITTING CACHE 'Physical' TO DATABASE > > > > > > > > SETTING TASK GETPHYSHIST > > > > > > > > SETTING STEP 0 > > > > TASK: GETPHYSHIST > > > > > > > > STARTING CACHE FOR PhysicalAction > > > > "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > > > > ssphys: unknown action 21 at offset 0x1a0 > > > > Try `ssphys --help` for more information > > > > ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info > -ewindows-1252 > > "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > > > > at vss2svn.pl line 1251 > > > > > > > > > > > > I have a lot of time this kind of error (ssphys: unknown action 21 at > offset > > 0x1a0) > > > > > > > > And my execution finish like that: > > > > > > > > WARNING: control character 0x16 in text input at character 206 > > > > > > > > not well-formed (invalid token) at line 38, column 106, byte 1552 at > > /PerlApp/XML/Parser.pm line 187 > > > > > > > > > > > > > > > > > > > > > > > > > > > > With the version 0.11.0 alpha 1 I have a lot of this kind of warning : > > > > > > > > WARNING: Unknown action 'ArchiveVersionsofFile' > > > > "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQDAAAAA" > > > > > > > > WARNING: Unknown action 'ArchiveVersionsofFile' > > > > "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQEAAAAA" > > > > ssphys: unknown action 21 at offset 0x1a0 > > > > Try `ssphys --help` for more information > > > > ERROR -- FAILED with non-zero exit status 255 > > > > at vss2svn.pl line 1092 > > > > "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQFAAAAA" > > > > > > > > Then it finish the execution but lost a lot of files directories. > > > > > > > > Like this: > > > > > > > > at vss2svn.pl line 851 > > > > Parent path missing while trying to add item > > '/orphaned/_DOIAAAAA/deliverycursor/' at revision 42: adding missing > parents > > > > > > > > > > > > > > > > Who can help me ? > > > > > > > > I have done vssbinPath/ANALYZE.EXE -banalyze-results-fix -f -d -v4 -i- > data > > and there are no more errors. > > > > Thank?s > > > > Thomas > > > > > > ---------------------------------------------------------------- > > This message was sent using IMP, the Internet Messaging Program. > > > > _______________________________________________ > > vss2svn-users mailing list > > Project homepage: > > http://www.pumacode.org/projects/vss2svn/ > > Subscribe/Unsubscribe/Admin: > > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > > Mailing list web interface (with searchable archives): > > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > > > > > > > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From vss2svn at nogga.de Tue Jun 26 05:44:50 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 26 05:46:32 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <4680D30F.4050003@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> Message-ID: <4680E012.3020402@nogga.de> > I tried your suggestion and from what I can see it looks good. But I > don't use a mixed setup, so someone else should try this out. Ok, one major drawback: we have to compute the size of the converted files for the Content-length: attribute. Otherwise the import will fail. So we have to do the conversion into a temporary file upfront and then copy the temporary file. This will also be necessary for the MD5 checksum. Dirk From toby at etjohnson.us Tue Jun 26 09:14:07 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jun 26 09:14:09 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <4680E012.3020402@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> <4680E012.3020402@nogga.de> Message-ID: <4681111F.1060401@etjohnson.us> Dirk wrote: > >> I tried your suggestion and from what I can see it looks good. But I >> don't use a mixed setup, so someone else should try this out. > > Ok, one major drawback: we have to compute the size of the converted > files for the Content-length: attribute. Otherwise the import will > fail. So we have to do the conversion into a temporary file upfront > and then copy the temporary file. This will also be necessary for the > MD5 checksum. I think we should be able to simply keep a running sum of the content-length as we output each line. I can't think of any way around computing MD5 sums aside from a temp file, but have we determined that it's really necessary to compute those? It just seems a bit of a shame to slow everything down with a tempfile, although with all the other disk access already it probably won't be too much of an incremental change... toby From vss2svn at nogga.de Tue Jun 26 10:44:56 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 26 10:46:28 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <4681111F.1060401@etjohnson.us> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> <4680E012.3020402@nogga.de> <4681111F.1060401@etjohnson.us> Message-ID: <46812668.9050906@nogga.de> > > I think we should be able to simply keep a running sum of the > content-length as we output each line. I can't think of any way around > computing MD5 sums aside from a temp file, but have we determined that > it's really necessary to compute those? It just seems a bit of a shame > to slow everything down with a tempfile, although with all the other > disk access already it probably won't be too much of an incremental > change... In theory this is correct, but ... you have to output the "Content-length" before the content, so you have the following possibilities * stream content into memory, compute the length, output the length and then the content from memory * stream content into memory, compute the length, drop the content, output the length and then read/write everything again * use an external tmp file ;-) I don't know, whether this "big file" issue is really a problem for "native-encoded", non binary files. So probably the first option is a practicable way. Dirk From toby at etjohnson.us Tue Jun 26 13:27:54 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jun 26 13:27:58 2007 Subject: vss2svn --auto-props do not convert "svn:eol-style=native" files correctly? In-Reply-To: <46812668.9050906@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> <4680E012.3020402@nogga.de> <4681111F.1060401@etjohnson.us> <46812668.9050906@nogga.de> Message-ID: <46814C9A.50501@etjohnson.us> Dirk wrote: > >> >> I think we should be able to simply keep a running sum of the >> content-length as we output each line. I can't think of any way >> around computing MD5 sums aside from a temp file, but have we >> determined that it's really necessary to compute those? It just seems >> a bit of a shame to slow everything down with a tempfile, although >> with all the other disk access already it probably won't be too much >> of an incremental change... > > In theory this is correct, but ... you have to output the > "Content-length" before the content, so you have the following > possibilities > > * stream content into memory, compute the length, output the length > and then the content from memory > * stream content into memory, compute the length, drop the content, > output the length and then read/write everything again > * use an external tmp file ;-) > > I don't know, whether this "big file" issue is really a problem for > "native-encoded", non binary files. So probably the first option is a > practicable way. True, that is quite the catch-22. But as soon as we implement the first option, someone will come along with a repository full of 500MB text files. :) Sounds like going the tmp file route is probably the sanest, and it will also allow MD5 creation (although I think that should be an option which defaults to off, since I'm guessing it would slow everything down quite a bit). toby From vss2svn at nogga.de Tue Jun 26 18:30:32 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jun 26 18:35:00 2007 Subject: MD5, UUID In-Reply-To: <46814C9A.50501@etjohnson.us> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> <4680E012.3020402@nogga.de> <4681111F.1060401@etjohnson.us> <46812668.9050906@nogga.de> <46814C9A.50501@etjohnson.us> Message-ID: <46819388.2050208@nogga.de> Hello Toby, besides some other changes I have added UUID and MD5 checksum generation. The later is optional. Nevertheless the new code needs two more modules Digest::MD5 Data::UUID Is it ok to check in, or will the build procedure break? Best regards Dirk From toby at etjohnson.us Tue Jun 26 22:51:34 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jun 26 22:51:38 2007 Subject: MD5, UUID In-Reply-To: <46819388.2050208@nogga.de> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> <4680E012.3020402@nogga.de> <4681111F.1060401@etjohnson.us> <46812668.9050906@nogga.de> <46814C9A.50501@etjohnson.us> <46819388.2050208@nogga.de> Message-ID: <4681D0B6.6050701@etjohnson.us> Dirk wrote: > Hello Toby, > > besides some other changes I have added UUID and MD5 checksum > generation. The later is optional. Nevertheless the new code needs two > more modules > > Digest::MD5 > Data::UUID > > Is it ok to check in, or will the build procedure break? Wow, you've been very busy tonight! Thanks for going through all those... I have installed the 2 modules you mention on my build machine, and added them to the build file to make sure they get bundled in. toby From thomas.franconville at sysdeo.fr Wed Jun 27 03:05:59 2007 From: thomas.franconville at sysdeo.fr (Thomas Franconville) Date: Wed Jun 27 03:38:32 2007 Subject: ssphys: unknown action 21 at offset 0x1a0 In-Reply-To: <46814628.1030905@nogga.de> References: <1182843459.4680c243d12bd@webmail.sysdeo.fr> <4680CD09.6010609@nogga.de> <1182867952.468121f07d906@webmail.sysdeo.fr> <46814628.1030905@nogga.de> Message-ID: <1182927959.46820c57399a4@webmail.sysdeo.fr> Hello Dirk, Hi all, Yes, the firsts actions we have is called "Versions archiv?es de ..." like "archived version of a file". It may correspond to the restore activity you are saying. We have then a "?tiquet?" like "labelled" And then "Archiv?e dans" like "archived in" that should correspond to a commit. And so on. So what about the restore activity and the vss2svn project ? I think, we have this restore activity because we have a vss project by branch. and so the restore, is the start of the new vss project. it is quite like an import or even an add. is it possible to use this kind of activities instead ? Cheers Thomas Selon Dirk : > Hello Thomas, > > thanks, that was exactly the same, what I would suggest to you. > > If you open the vss explorer and generate the history for the root > project. What are the first two actions? I would expect some kind of > restore activity. > > Best regards > Dirk > > Thomas Franconville schrieb: > > You will find a vss data file that give this error. > > I prefer sending it to you directly rather than with the mailing list. > > If you can find something with... > > > > D:\Program\vss2svn>ssphys info D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA > > > > > > > > > > 46 > > .A > > D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA.A > > 1 > > 4 > > 4 > > > > > > > > > > ssphys: unknown action 21 at offset 0x1a0 > > Try `ssphys --help` for more information > > > > Selon Dirk : > > > > > >> Hello Thomas, > >> > >> this means, that you used some type of vss activity, that is not decoded > >> yet. In other words, you did something, that nobody else who used the > >> converter did up to today ;-) > >> > >> If your archive is still intact, can you please tell me the list of > >> activities on your root project? There must be something unusual in this > >> history. Not common tasks, like create project, create file, rename > >> delete and so on. > >> > >> > >> Dirk > >> > >> > >> Thomas Franconville schrieb: > >> > >>> Hello, > >>> > >>> > >>> > >>> I have problems with converting a vss project with the nightly build > >>> > >> version of > >> > >>> vss2svn > >>> > >>> > >>> > >>> > >>> > >>> D:\Program\vss2svn>vss2svn.exe --verbose --vssdir D:\VSS\DELIVERY-3.3 > >>> > >>> Adding > >>> > >> 'C:/DOCUME~1/TFRANC~1/LOCALS~1/Temp/pdk-tfranconville-2356/encodings' to > >> > >>> encodings file path > >>> > >>> Connecting to database ./_vss2svn/vss_data.db > >>> > >>> > >>> > >>> ======== VSS2SVN ======== > >>> > >>> BEGINNING CONVERSION... > >>> > >>> Start Time : Mon Jun 25 15:45:00 2007 > >>> > >>> > >>> > >>> > >>> > >>> VSS Dir : D:\VSS\DELIVERY-3.3SETTING STEP 0 > >>> > >>> TASK: GETPHYSHIST > >>> > >>> > >>> > >>> STARTING CACHE FOR PhysicalAction > >>> > >>> "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > >>> > >>> ssphys: unknown action 21 at offset 0x1a0 > >>> > >>> Try `ssphys --help` for more information > >>> > >>> ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info > >>> > >> -ewindows-1252 > >> > >>> "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > >>> > >>> at vss2svn.pl line 1251 > >>> > >>> Temp Dir : ./_vss2svn > >>> > >>> Dumpfile : vss2svn-dumpfile.dat > >>> > >>> VSS Encoding : windows-1252 > >>> > >>> > >>> > >>> VSS2SVN ver : 0.11.0-nightly.315 > >>> > >>> SSPHYS exe : ssphys > >>> > >>> SSPHYS ver : 0.22.0.275 > >>> > >>> XML Parser : XML::SAX::Expat > >>> > >>> > >>> > >>> TASK: INIT > >>> > >>> > >>> > >>> SETTING TASK LOADVSSNAMES > >>> > >>> > >>> > >>> SETTING STEP 0 > >>> > >>> TASK: LOADVSSNAMES > >>> > >>> "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/names.dat" > >>> > >>> > >>> > >>> STARTING CACHE FOR NameLookup > >>> > >>> > >>> > >>> > >>> > >>> COMMITTING CACHE 'NameLookup' TO DATABASE > >>> > >>> > >>> > >>> SETTING TASK FINDDBFILES > >>> > >>> > >>> > >>> SETTING STEP 0 > >>> > >>> TASK: FINDDBFILES > >>> > >>> > >>> > >>> STARTING CACHE FOR Physical > >>> > >>> > >>> > >>> > >>> > >>> COMMITTING CACHE 'Physical' TO DATABASE > >>> > >>> > >>> > >>> SETTING TASK GETPHYSHIST > >>> > >>> > >>> > >>> SETTING STEP 0 > >>> > >>> TASK: GETPHYSHIST > >>> > >>> > >>> > >>> STARTING CACHE FOR PhysicalAction > >>> > >>> "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > >>> > >>> ssphys: unknown action 21 at offset 0x1a0 > >>> > >>> Try `ssphys --help` for more information > >>> > >>> ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info > >>> > >> -ewindows-1252 > >> > >>> "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > >>> > >>> at vss2svn.pl line 1251 > >>> > >>> > >>> > >>> > >>> > >>> I have a lot of time this kind of error (ssphys: unknown action 21 at > >>> > >> offset > >> > >>> 0x1a0) > >>> > >>> > >>> > >>> And my execution finish like that: > >>> > >>> > >>> > >>> WARNING: control character 0x16 in text input at character 206 > >>> > >>> > >>> > >>> not well-formed (invalid token) at line 38, column 106, byte 1552 at > >>> /PerlApp/XML/Parser.pm line 187 > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> With the version 0.11.0 alpha 1 I have a lot of this kind of warning : > >>> > >>> > >>> > >>> WARNING: Unknown action 'ArchiveVersionsofFile' > >>> > >>> "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQDAAAAA" > >>> > >>> > >>> > >>> WARNING: Unknown action 'ArchiveVersionsofFile' > >>> > >>> "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQEAAAAA" > >>> > >>> ssphys: unknown action 21 at offset 0x1a0 > >>> > >>> Try `ssphys --help` for more information > >>> > >>> ERROR -- FAILED with non-zero exit status 255 > >>> > >>> at vss2svn.pl line 1092 > >>> > >>> "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQFAAAAA" > >>> > >>> > >>> > >>> Then it finish the execution but lost a lot of files directories. > >>> > >>> > >>> > >>> Like this: > >>> > >>> > >>> > >>> at vss2svn.pl line 851 > >>> > >>> Parent path missing while trying to add item > >>> '/orphaned/_DOIAAAAA/deliverycursor/' at revision 42: adding missing > >>> > >> parents > >> > >>> > >>> > >>> > >>> > >>> > >>> Who can help me ? > >>> > >>> > >>> > >>> I have done vssbinPath/ANALYZE.EXE -banalyze-results-fix -f -d -v4 -i- > >>> > >> data > >> > >>> and there are no more errors. > >>> > >>> Thank?s > >>> > >>> Thomas > >>> > >>> > >>> ---------------------------------------------------------------- > >>> This message was sent using IMP, the Internet Messaging Program. > >>> > >>> _______________________________________________ > >>> vss2svn-users mailing list > >>> Project homepage: > >>> http://www.pumacode.org/projects/vss2svn/ > >>> Subscribe/Unsubscribe/Admin: > >>> > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > >>> Mailing list web interface (with searchable archives): > >>> http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > >>> > >>> > >>> > >>> > >>> > >> _______________________________________________ > >> vss2svn-users mailing list > >> Project homepage: > >> http://www.pumacode.org/projects/vss2svn/ > >> Subscribe/Unsubscribe/Admin: > >> > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > >> Mailing list web interface (with searchable archives): > >> http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > >> > >> > >> > > > > > > > > > > ---------------------------------------------------------------- > > This message was sent using IMP, the Internet Messaging Program. > > > > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From vss2svn at nogga.de Wed Jun 27 04:36:44 2007 From: vss2svn at nogga.de (Dirk) Date: Wed Jun 27 04:38:39 2007 Subject: MD5, UUID In-Reply-To: <4681D0B6.6050701@etjohnson.us> References: <464CCD74.6020301@hrl.com> <465725DC.9000102@etjohnson.us> <46785580.6020902@nogga.de> <467ED29F.70308@etjohnson.us> <467F75DB.7060408@nogga.de> <467F8AD4.2020509@nogga.de> <468061FE.10905@etjohnson.us> <4680D30F.4050003@nogga.de> <4680E012.3020402@nogga.de> <4681111F.1060401@etjohnson.us> <46812668.9050906@nogga.de> <46814C9A.50501@etjohnson.us> <46819388.2050208@nogga.de> <4681D0B6.6050701@etjohnson.us> Message-ID: <4682219C.7010605@nogga.de> > Wow, you've been very busy tonight! Thanks for going through all > those... I have installed the 2 modules you mention on my build > machine, and added them to the build file to make sure they get > bundled in. No problem, after some long period of real work overload I will finally convert our archive in the next two weeks. So this is the last cleanup ;-) I'm currently looking into to more changes: * for PIN activities use merge instead of copy * automatically relocate labels and projects into different target locations in order to clean up the archive. I have tons of labels and I want to have them grouped. Currently I have a shadow subversion copy of my VSS archive with a nightly build, and I really like the new possibilities to dig through my data ;-) Best regards Dirk From thomas.franconville at sysdeo.fr Wed Jun 27 04:06:52 2007 From: thomas.franconville at sysdeo.fr (Thomas Franconville) Date: Wed Jun 27 04:39:26 2007 Subject: ssphys: unknown action 21 at offset 0x1a0 In-Reply-To: <1182927959.46820c57399a4@webmail.sysdeo.fr> References: <1182843459.4680c243d12bd@webmail.sysdeo.fr> <4680CD09.6010609@nogga.de> <1182867952.468121f07d906@webmail.sysdeo.fr> <46814628.1030905@nogga.de> <1182927959.46820c57399a4@webmail.sysdeo.fr> Message-ID: <1182931612.46821a9ca0b8c@webmail.sysdeo.fr> I have a small projet test given in attached file. Here is the history. I began with a first project called TEST. do some stuff into TEST Label it to "branch test" Copy the project TEST to TEST2 Archived TEST2 until the label "branch test" Do some stuff into TEST2 vss2svn goes to the end of it's process, but do quite the same errors and is losing the paths of files. ssphys: unknown action 21 at offset 0x1a0 Try `ssphys --help` for more information ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 "D:\VSS\TEST2/data/A/AAAAAAAA") at vss2svn.pl line 1251 WARNING: control character 0x12 in text input at character 0 ssphys: unknown action 21 at offset 0x1a0 Try `ssphys --help` for more information ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 "D:\VSS\TEST2/data/D/DAAAAAAA") at vss2svn.pl line 1251 WARNING: control character 0x12 in text input at character 0 WARNING: control character 0x12 in text input at character 0 ERROR -- Attempt to move unknown item 'BAAAAAAA': at vss2svn.pl line 847 ERROR -- Attempt to move unknown item 'CAAAAAAA': at vss2svn.pl line 847 ERROR -- Attempt to move unknown item 'EAAAAAAA': at vss2svn.pl line 847 ERROR -- Attempt to move unknown item 'FAAAAAAA': at vss2svn.pl line 847 ERROR -- Attempt to move unknown item 'GAAAAAAA': at vss2svn.pl line 847 ERROR -- Attempt to commit unknown item 'FAAAAAAA': at vss2svn.pl line 847 ERROR -- Parent path missing while trying to add item '/orphaned/_HAAAAAAA/tata.txt' at revision 1: adding missing parents at vss2svn.pl line 1002 ERROR -- Parent path missing while trying to add item '/orphaned/_IAAAAAAA/tutu.txt' at revision 1: adding missing parents at vss2svn.pl line 1002 Selon Thomas Franconville : > Hello Dirk, Hi all, > > Yes, the firsts actions we have is called "Versions archiv?es de ..." like > "archived version of a file". It may correspond to the restore activity you > are > saying. > We have then a "?tiquet?" like "labelled" > And then "Archiv?e dans" like "archived in" that should correspond to a > commit. > And so on. > > So what about the restore activity and the vss2svn project ? > I think, we have this restore activity because we have a vss project by > branch. > and so the restore, is the start of the new vss project. it is quite like an > import or even an add. is it possible to use this kind of activities instead > ? > > Cheers > > Thomas > > Selon Dirk : > > > Hello Thomas, > > > > thanks, that was exactly the same, what I would suggest to you. > > > > If you open the vss explorer and generate the history for the root > > project. What are the first two actions? I would expect some kind of > > restore activity. > > > > Best regards > > Dirk > > > > Thomas Franconville schrieb: > > > You will find a vss data file that give this error. > > > I prefer sending it to you directly rather than with the mailing list. > > > If you can find something with... > > > > > > D:\Program\vss2svn>ssphys info D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA > > > > > > > > > > > > > > > 46 > > > .A > > > > D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA.A > > > 1 > > > 4 > > > 4 > > > > > > > > > > > > > > > ssphys: unknown action 21 at offset 0x1a0 > > > Try `ssphys --help` for more information > > > > > > Selon Dirk : > > > > > > > > >> Hello Thomas, > > >> > > >> this means, that you used some type of vss activity, that is not decoded > > >> yet. In other words, you did something, that nobody else who used the > > >> converter did up to today ;-) > > >> > > >> If your archive is still intact, can you please tell me the list of > > >> activities on your root project? There must be something unusual in this > > >> history. Not common tasks, like create project, create file, rename > > >> delete and so on. > > >> > > >> > > >> Dirk > > >> > > >> > > >> Thomas Franconville schrieb: > > >> > > >>> Hello, > > >>> > > >>> > > >>> > > >>> I have problems with converting a vss project with the nightly build > > >>> > > >> version of > > >> > > >>> vss2svn > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> D:\Program\vss2svn>vss2svn.exe --verbose --vssdir D:\VSS\DELIVERY-3.3 > > >>> > > >>> Adding > > >>> > > >> 'C:/DOCUME~1/TFRANC~1/LOCALS~1/Temp/pdk-tfranconville-2356/encodings' to > > >> > > >>> encodings file path > > >>> > > >>> Connecting to database ./_vss2svn/vss_data.db > > >>> > > >>> > > >>> > > >>> ======== VSS2SVN ======== > > >>> > > >>> BEGINNING CONVERSION... > > >>> > > >>> Start Time : Mon Jun 25 15:45:00 2007 > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> VSS Dir : D:\VSS\DELIVERY-3.3SETTING STEP 0 > > >>> > > >>> TASK: GETPHYSHIST > > >>> > > >>> > > >>> > > >>> STARTING CACHE FOR PhysicalAction > > >>> > > >>> "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > > >>> > > >>> ssphys: unknown action 21 at offset 0x1a0 > > >>> > > >>> Try `ssphys --help` for more information > > >>> > > >>> ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info > > >>> > > >> -ewindows-1252 > > >> > > >>> "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > > >>> > > >>> at vss2svn.pl line 1251 > > >>> > > >>> Temp Dir : ./_vss2svn > > >>> > > >>> Dumpfile : vss2svn-dumpfile.dat > > >>> > > >>> VSS Encoding : windows-1252 > > >>> > > >>> > > >>> > > >>> VSS2SVN ver : 0.11.0-nightly.315 > > >>> > > >>> SSPHYS exe : ssphys > > >>> > > >>> SSPHYS ver : 0.22.0.275 > > >>> > > >>> XML Parser : XML::SAX::Expat > > >>> > > >>> > > >>> > > >>> TASK: INIT > > >>> > > >>> > > >>> > > >>> SETTING TASK LOADVSSNAMES > > >>> > > >>> > > >>> > > >>> SETTING STEP 0 > > >>> > > >>> TASK: LOADVSSNAMES > > >>> > > >>> "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/names.dat" > > >>> > > >>> > > >>> > > >>> STARTING CACHE FOR NameLookup > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> COMMITTING CACHE 'NameLookup' TO DATABASE > > >>> > > >>> > > >>> > > >>> SETTING TASK FINDDBFILES > > >>> > > >>> > > >>> > > >>> SETTING STEP 0 > > >>> > > >>> TASK: FINDDBFILES > > >>> > > >>> > > >>> > > >>> STARTING CACHE FOR Physical > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> COMMITTING CACHE 'Physical' TO DATABASE > > >>> > > >>> > > >>> > > >>> SETTING TASK GETPHYSHIST > > >>> > > >>> > > >>> > > >>> SETTING STEP 0 > > >>> > > >>> TASK: GETPHYSHIST > > >>> > > >>> > > >>> > > >>> STARTING CACHE FOR PhysicalAction > > >>> > > >>> "ssphys" info -ewindows-1252 "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA" > > >>> > > >>> ssphys: unknown action 21 at offset 0x1a0 > > >>> > > >>> Try `ssphys --help` for more information > > >>> > > >>> ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info > > >>> > > >> -ewindows-1252 > > >> > > >>> "D:\VSS\DELIVERY-3.3/data/A/AAAAAAAA") > > >>> > > >>> at vss2svn.pl line 1251 > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> I have a lot of time this kind of error (ssphys: unknown action 21 at > > >>> > > >> offset > > >> > > >>> 0x1a0) > > >>> > > >>> > > >>> > > >>> And my execution finish like that: > > >>> > > >>> > > >>> > > >>> WARNING: control character 0x16 in text input at character 206 > > >>> > > >>> > > >>> > > >>> not well-formed (invalid token) at line 38, column 106, byte 1552 at > > >>> /PerlApp/XML/Parser.pm line 187 > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> With the version 0.11.0 alpha 1 I have a lot of this kind of warning : > > >>> > > >>> > > >>> > > >>> WARNING: Unknown action 'ArchiveVersionsofFile' > > >>> > > >>> "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQDAAAAA" > > >>> > > >>> > > >>> > > >>> WARNING: Unknown action 'ArchiveVersionsofFile' > > >>> > > >>> "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQEAAAAA" > > >>> > > >>> ssphys: unknown action 21 at offset 0x1a0 > > >>> > > >>> Try `ssphys --help` for more information > > >>> > > >>> ERROR -- FAILED with non-zero exit status 255 > > >>> > > >>> at vss2svn.pl line 1092 > > >>> > > >>> "ssphys" info "D:\VSS\DELIVERY-3.3/data/W/WQFAAAAA" > > >>> > > >>> > > >>> > > >>> Then it finish the execution but lost a lot of files directories. > > >>> > > >>> > > >>> > > >>> Like this: > > >>> > > >>> > > >>> > > >>> at vss2svn.pl line 851 > > >>> > > >>> Parent path missing while trying to add item > > >>> '/orphaned/_DOIAAAAA/deliverycursor/' at revision 42: adding missing > > >>> > > >> parents > > >> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> Who can help me ? > > >>> > > >>> > > >>> > > >>> I have done vssbinPath/ANALYZE.EXE -banalyze-results-fix -f -d -v4 -i- > > >>> > > >> data > > >> > > >>> and there are no more errors. > > >>> > > >>> Thank?s > > >>> > > >>> Thomas > > >>> > > >>> > > >>> ---------------------------------------------------------------- > > >>> This message was sent using IMP, the Internet Messaging Program. > > >>> > > >>> _______________________________________________ > > >>> vss2svn-users mailing list > > >>> Project homepage: > > >>> http://www.pumacode.org/projects/vss2svn/ > > >>> Subscribe/Unsubscribe/Admin: > > >>> > > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > > >>> Mailing list web interface (with searchable archives): > > >>> http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > >>> > > >>> > > >>> > > >>> > > >>> > > >> _______________________________________________ > > >> vss2svn-users mailing list > > >> Project homepage: > > >> http://www.pumacode.org/projects/vss2svn/ > > >> Subscribe/Unsubscribe/Admin: > > >> > > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > > >> Mailing list web interface (with searchable archives): > > >> http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > >> > > >> > > >> > > > > > > > > > > > > > > > ---------------------------------------------------------------- > > > This message was sent using IMP, the Internet Messaging Program. > > > > > > > > > > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. -------------- next part -------------- A non-text attachment was scrubbed... Name: TEST2.zip Type: application/zip Size: 15600 bytes Desc: not available Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070627/8d5cacc6/TEST2-0001.zip From vss2svn at nogga.de Fri Jun 29 04:58:19 2007 From: vss2svn at nogga.de (Dirk) Date: Fri Jun 29 05:00:26 2007 Subject: ssphys: unknown action 21 at offset 0x1a0 In-Reply-To: <1182931612.46821a9ca0b8c@webmail.sysdeo.fr> References: <1182843459.4680c243d12bd@webmail.sysdeo.fr> <4680CD09.6010609@nogga.de> <1182867952.468121f07d906@webmail.sysdeo.fr> <46814628.1030905@nogga.de> <1182927959.46820c57399a4@webmail.sysdeo.fr> <1182931612.46821a9ca0b8c@webmail.sysdeo.fr> Message-ID: <4684C9AB.5050101@nogga.de> Hello Thomas, the problem is a little difficult. I have modified the vss decoder to read this action, but the results whenn running the converter are still broken. The problem is, that the timeline is completely out of sync. This new action is called "archived version of a file". It is special, since only one version at a specific state of each item in the archive is archived. The restore creates only this version of the item. The problem is, that timestamp for these restored items are (probably) set to the last version of the item, so it is not possible to order the restored items by timestamp, since this doesn't give you the correct order of events.. Funny enough ss itself can't decode the full history, otherwise I would have posted a dump of "ss history". If you look into the history with the ssexp browser, you will e.g. note that the items within the root project are archived with a timestamp prior to the root project itself. Due to this problem all items are mapped to the orphaned folder and they can not restored back into the correct pathes. The only possibility is to detect some hierarchy via the "PF" records, but this is a completly different beast and not easily doable in the current converter. So I think, you are a little lost with your archive in this situation. BEst regards Dirk Thomas Franconville schrieb: > I have a small projet test given in attached file. > Here is the history. > I began with a first project called TEST. > do some stuff into TEST > Label it to "branch test" > Copy the project TEST to TEST2 > Archived TEST2 until the label "branch test" > Do some stuff into TEST2 > > vss2svn goes to the end of it's process, but do quite the same errors and is > losing the paths of files. > > ssphys: unknown action 21 at offset 0x1a0 > Try `ssphys --help` for more information > ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 > "D:\VSS\TEST2/data/A/AAAAAAAA") > at vss2svn.pl line 1251 > WARNING: control character 0x12 in text input at character 0 > ssphys: unknown action 21 at offset 0x1a0 > Try `ssphys --help` for more information > ERROR -- FAILED with non-zero exit status 255 (cmd: "ssphys" info -ewindows-1252 > "D:\VSS\TEST2/data/D/DAAAAAAA") > at vss2svn.pl line 1251 > WARNING: control character 0x12 in text input at character 0 > WARNING: control character 0x12 in text input at character 0 > ERROR -- Attempt to move unknown item 'BAAAAAAA': > > at vss2svn.pl line 847 > ERROR -- Attempt to move unknown item 'CAAAAAAA': > > at vss2svn.pl line 847 > ERROR -- Attempt to move unknown item 'EAAAAAAA': > > at vss2svn.pl line 847 > ERROR -- Attempt to move unknown item 'FAAAAAAA': > > at vss2svn.pl line 847 > ERROR -- Attempt to move unknown item 'GAAAAAAA': > > at vss2svn.pl line 847 > ERROR -- Attempt to commit unknown item 'FAAAAAAA': > > at vss2svn.pl line 847 > ERROR -- Parent path missing while trying to add item > '/orphaned/_HAAAAAAA/tata.txt' at revision 1: adding missing parents > at vss2svn.pl line 1002 > ERROR -- Parent path missing while trying to add item > '/orphaned/_IAAAAAAA/tutu.txt' at revision 1: adding missing parents > at vss2svn.pl line 1002 > > Selon Thomas Franconville : > > >> Hello Dirk, Hi all, >> >> Yes, the firsts actions we have is called "Versions archiv?es de ..." like >> "archived version of a file". It may correspond to the restore activity you >> are >> saying. >> We have then a "?tiquet?" like "labelled" >> And then "Archiv?e dans" like "archived in" that should correspond to a >> commit. >> And so on. >> >> So what about the restore activity and the vss2svn project ? >> I think, we have this restore activity because we have a vss project by >> branch. >> and so the restore, is the start of the new vss project. it is quite like an >> import or even an add. is it possible to use this kind of activities instead >> ? >> >> Cheers >> >> Thomas >>