From oavtal at bezeqint.net Tue Jan 2 16:26:01 2007 From: oavtal at bezeqint.net (Ori Avtalion) Date: Tue Jan 2 15:26:06 2007 Subject: Mixed encodings in dump file Message-ID: <459ACDE9.9070009@bezeqint.net> Hi, Using vss2svn 0.11.0-alpha1 on a repository with hebrew text files and hebrew log messages, I received a strange svn dump file: The patches were in the correct cp1255 encoding, but the log messages were in utf-8. The result is that 'svnadmin load' produced log messages with the wrong encoding. Any idea what I should do? Regarding ticket #26 Is it correct to convert files I wrote in a specific encoding to utf-8 without regard? Specifically, I sadly need to store some VB6 source files, and the VB6 IDE doesn't like UTF-8. Thanks, Ori From shiva at sewingwitch.com Tue Jan 2 16:46:38 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Tue Jan 2 15:46:46 2007 Subject: Mixed encodings in dump file In-Reply-To: <459ACDE9.9070009@bezeqint.net> References: <459ACDE9.9070009@bezeqint.net> Message-ID: --On Tuesday, January 02, 2007 11:26 PM +0200 Ori Avtalion wrote: > Regarding ticket #26 > Is it correct to convert files I wrote in a specific encoding to utf-8 > without regard? Specifically, I sadly need to store some VB6 source > files, and the VB6 IDE doesn't like UTF-8. Subversion stores everything on the server (and in dump files) in Unicode. The client is responsible for converting to the local encoding on checkout. From oavtal at bezeqint.net Thu Jan 4 02:02:48 2007 From: oavtal at bezeqint.net (Ori Avtalion) Date: Thu Jan 4 02:02:51 2007 Subject: Mixed encodings in dump file Message-ID: Hi again, I was a bit confused earlier with the encodings. vss2svn thinks the comments in my vss repository are encoded in latin1, but since it's hebrew they're encoded in cp-1255. vss2svn then converts them from latin1 to utf-8, which makes the comments unreadable. To test, I isolated the a log message in the dump file, converted it from utf-8 to latin1, and then from cp1255 to utf-8. The log message is now encoded correctly in utf-8. Is it possible to add a flag to vss2svn that specifies the correct character encoding? From shiva at sewingwitch.com Thu Jan 4 03:03:54 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Thu Jan 4 03:04:01 2007 Subject: Mixed encodings in dump file In-Reply-To: References: Message-ID: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> --On Thursday, January 04, 2007 10:02 AM +0200 Ori Avtalion wrote: > Is it possible to add a flag to vss2svn that specifies the correct > character encoding? We had discussed doing this. (See the list archives.) The issue is in ssphys, the program that converts the binary VSS files to XML. It's hard-coded to claim the encoding of the files. You can edit the line that installs the encoding in the XML file to your desired encoding. A patch to make this a command line parameter would be welcome. From toby at etjohnson.us Thu Jan 4 08:52:44 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu Jan 4 08:52:49 2007 Subject: Mixed encodings in dump file In-Reply-To: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> Message-ID: <459D14BC.5050302@etjohnson.us> Kenneth Porter wrote: > --On Thursday, January 04, 2007 10:02 AM +0200 Ori Avtalion > wrote: > >> Is it possible to add a flag to vss2svn that specifies the correct >> character encoding? > > We had discussed doing this. (See the list archives.) The issue is in > ssphys, the program that converts the binary VSS files to XML. It's > hard-coded to claim the encoding of the files. You can edit the line > that installs the encoding in the XML file to your desired encoding. A > patch to make this a command line parameter would be welcome. Kenneth, I must admit that I am very inexperienced with issues regarding code pages and converting among encodings... according to your description, is ticket 26 still valid? It sounds like the XML parser will "do the right thing" as long as the correct encoding is written to the XML files? toby From vss2svn at nogga.de Thu Jan 4 10:17:51 2007 From: vss2svn at nogga.de (Dirk) Date: Thu Jan 4 10:15:47 2007 Subject: Mixed encodings in dump file In-Reply-To: <459D14BC.5050302@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> Message-ID: <459D28AF.9050408@nogga.de> > Kenneth, I must admit that I am very inexperienced with issues > regarding code pages and converting among encodings... according to > your description, is ticket 26 > still valid? It > sounds like the XML parser will "do the right thing" as long as the > correct encoding is written to the XML files? > Oh, this old devil jumps back into my neck ... First: vss itself does not contain any codepage information. Since every client will write the settings in its own codepage, you can have a mixed archive with different encodings, but you can not tell from the outside, which one is the correct one. E.g. consider two developers with different codepages working on the same archive. Both will write log messages in their own codepage, and therefor can not correctly decode the log messages from the other. And there is no way to prevent this. Second: You have to distinguish the version controlled files and the associated information like author and comment. No source control system will deal with the "encoding" of the stored file itself. You can consider this as black box data. Only comment and author information needs ot be encoded in the corret codepage. This is why you will see utf-8 encoded comments but codepage encoded data in the dumpfile. Third: There is a long outstanding problem to save all this information during the conversion in vss2svn. Initially I thought, that putting the right codepage into the header of the ssphys generated xml file will solve all problems, but this wasn't the case. So we are still looking for a good solution and there are a few workarounds. 1.) I'm not sure about the state of the Ken/Unicode [1] and Ken/ssphys-trusted-encoding branches [2] 2.) What happens if you patch this line[3] to change to your correct codepage > TiXmlDeclaration decl ("1.0", "windows-1252", ""); Best regards Dirk [1] http://www.pumacode.org/projects/vss2svn/browser/branches/Ken/Unicode [2] http://www.pumacode.org/projects/vss2svn/browser/branches/Ken/ssphys-trusted-encoding [3] http://www.pumacode.org/projects/vss2svn/browser/trunk/ssphys/SSPhys/Formatter.cpp#L57 From shiva at sewingwitch.com Thu Jan 4 10:36:35 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Thu Jan 4 10:36:42 2007 Subject: Mixed encodings in dump file In-Reply-To: <459D14BC.5050302@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> Message-ID: <80AFE6B50EB540B69047B003@[10.170.7.6]> --On Thursday, January 04, 2007 9:52 AM -0500 Toby Johnson wrote: > is ticket 26 still > valid? I abandoned that approach and used Dirk's solution #2 in his reply, to simply declare the generated XML to be in whatever encoding we guess it to be. In my case it was CP1252, and that's what's in my latest branch ("Ken/restore-widetime"). From toby at etjohnson.us Thu Jan 4 10:39:25 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu Jan 4 10:39:29 2007 Subject: Mixed encodings in dump file In-Reply-To: <459D28AF.9050408@nogga.de> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <459D28AF.9050408@nogga.de> Message-ID: <459D2DBD.90806@etjohnson.us> Dirk wrote: > >> Kenneth, I must admit that I am very inexperienced with issues >> regarding code pages and converting among encodings... according to >> your description, is ticket 26 >> still valid? It >> sounds like the XML parser will "do the right thing" as long as the >> correct encoding is written to the XML files? >> > > Oh, this old devil jumps back into my neck ... > > First: vss itself does not contain any codepage information. Since > every client will write the settings in its own codepage, you can have > a mixed archive with different encodings, but you can not tell from > the outside, which one is the correct one. E.g. consider two > developers with different codepages working on the same archive. Both > will write log messages in their own codepage, and therefor can not > correctly decode the log messages from the other. And there is no way > to prevent this. > > Second: You have to distinguish the version controlled files and the > associated information like author and comment. No source control > system will deal with the "encoding" of the stored file itself. You > can consider this as black box data. Only comment and author > information needs ot be encoded in the corret codepage. > This is why you will see utf-8 encoded comments but codepage encoded > data in the dumpfile. OK, thanks for helping explain the issue better for me. I think us Americans happen to have fewer troubles with encodings than the rest of the world! Since it seems that all of us are finding it harder to get time to work on this project, I think we should find a solution that works "pretty well" with the least effort possible, so users don't need to recompile ssphys to get a good conversion. I think defaulting to Windows-1252 with the option to manually specify otherwise sounds like the way to go... > 1.) I'm not sure about the state of the Ken/Unicode [1] and > Ken/ssphys-trusted-encoding branches [2] > > > 2.) What happens if you patch this line[3] to change to your correct > codepage > > TiXmlDeclaration decl ("1.0", "windows-1252", ""); If I understand Kenneth's reply, #2 works correctly and the XML parser will convert to the proper encoding. toby From toby at etjohnson.us Thu Jan 4 10:42:10 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu Jan 4 10:42:16 2007 Subject: Mixed encodings in dump file In-Reply-To: <80AFE6B50EB540B69047B003@[10.170.7.6]> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> Message-ID: <459D2E62.2030201@etjohnson.us> Kenneth Porter wrote: > --On Thursday, January 04, 2007 9:52 AM -0500 Toby Johnson > wrote: > >> is ticket 26 still >> valid? > > I abandoned that approach and used Dirk's solution #2 in his reply, to > simply declare the generated XML to be in whatever encoding we guess > it to be. In my case it was CP1252, and that's what's in my latest > branch ("Ken/restore-widetime"). OK, thanks... does the XML parser then take care of it automatically, without any need to actually read the PI from the XML file in the vss2svn script? The ticket mentioned something about from_to() calls but those seem to no longer be there... toby From shiva at sewingwitch.com Thu Jan 4 10:54:37 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Thu Jan 4 10:54:45 2007 Subject: Mixed encodings in dump file In-Reply-To: <459D2E62.2030201@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> Message-ID: <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> --On Thursday, January 04, 2007 11:42 AM -0500 Toby Johnson wrote: > OK, thanks... does the XML parser then take care of it automatically, > without any need to actually read the PI from the XML file in the vss2svn > script? The ticket mentioned something about from_to() calls but those > seem to no longer be there... Correct. The parser does the conversion to native Perl Unicode. Take a look at r262 for what happened to from_to(). The string objects for author and comment are actually UTF-8 but without the Perl internal markers to make them recognized as such, so the Encode::decode_utf8 sort of does for UTF-8 encoding what a C++ placement new does for object allocation. It says "this is already UTF-8, so set the internal bit to recognize that". This prevents double-encoding when the comment and author are written to the dump file. From toby at etjohnson.us Thu Jan 4 10:59:03 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu Jan 4 10:59:09 2007 Subject: Mixed encodings in dump file In-Reply-To: <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> Message-ID: <459D3257.6080703@etjohnson.us> Kenneth Porter wrote: > --On Thursday, January 04, 2007 11:42 AM -0500 Toby Johnson > wrote: > >> OK, thanks... does the XML parser then take care of it automatically, >> without any need to actually read the PI from the XML file in the >> vss2svn >> script? The ticket mentioned something about from_to() calls but those >> seem to no longer be there... > > Correct. The parser does the conversion to native Perl Unicode. > > Take a look at r262 for what happened to from_to(). > > The string objects for author and comment are actually UTF-8 but > without the Perl internal markers to make them recognized as such, so > the Encode::decode_utf8 sort of does for UTF-8 encoding what a C++ > placement new does for object allocation. It says "this is already > UTF-8, so set the internal bit to recognize that". This prevents > double-encoding when the comment and author are written to the dump file. Gotcha. I'll close ticket 26 and open another to add an encoding switch to ssphys. From oavtal at bezeqint.net Sat Jan 6 16:42:27 2007 From: oavtal at bezeqint.net (Ori Avtalion) Date: Sat Jan 6 17:42:28 2007 Subject: Usernames and case-sensitivity Message-ID: <45A025D3.9040204@bezeqint.net> Hi, I noticed users in VSS are case-insensitive and stored as upper-case (I tried renaming them to lower-case without success). Is this true? They are also stored as uppercase in the subversion dumpfile. I'd like to import them to subversion as lower-case. Is it possible? From oavtal at bezeqint.net Sat Jan 6 17:53:12 2007 From: oavtal at bezeqint.net (Ori Avtalion) Date: Sat Jan 6 18:53:26 2007 Subject: Mixed encodings in dump file In-Reply-To: <459D3257.6080703@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> Message-ID: <45A03668.8000403@bezeqint.net> Toby Johnson wrote: > > Gotcha. I'll close ticket 26 and open another to add an encoding switch > to ssphys. > I have written a patch for Since I have technical problems at work with running vss2svn from the source, I cannot test it with my VSS repository. I know this isn't a healthy way to submit patches, but I think it will at least help speed the ticket along, and since I made changes to ssphys it can basically be tested with a regular latin-codepage VSS repository. There is a chance it may not run at all, though :) I've added an encoding flag to the xml formatter of ssphys with a default value of windows-1252, and a matching flag for vss2svn.pl (also with a default). Regarding DoSsCMD() in vss2svn.pl, the character range removed from the output is correct for *most* windows codepages, since all of them re-define just the lower part of windows-1255. Arabic, for example, has rarely used characters in 0x8D and 0x90 -Ori -------------- next part -------------- A non-text attachment was scrubbed... Name: vss-encoding.patch Type: text/x-patch Size: 5174 bytes Desc: not available Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070107/644c0910/vss-encoding.bin From toby at etjohnson.us Sat Jan 6 18:03:23 2007 From: toby at etjohnson.us (Toby Johnson) Date: Sat Jan 6 19:03:41 2007 Subject: Usernames and case-sensitivity In-Reply-To: <45A025D3.9040204@bezeqint.net> References: <45A025D3.9040204@bezeqint.net> Message-ID: <45A038CB.60909@etjohnson.us> Ori Avtalion wrote: > Hi, > > I noticed users in VSS are case-insensitive and stored as upper-case (I > tried renaming them to lower-case without success). Is this true? > They are not stored as all-uppercase in VSS, although if you initially named them that way, VSS may not consider it a rename if you change only the case. Perhaps it would work if you changed the name to something else, then back to the desired case. > They are also stored as uppercase in the subversion dumpfile. > I'd like to import them to subversion as lower-case. Is it possible? > If you didn't want to change them in VSS you could force them all to lowercase by modifying line 94 of Vss2Svn/Dumpfile.pm: Index: Dumpfile.pm =================================================================== --- Dumpfile.pm (revision 272) +++ Dumpfile.pm (working copy) @@ -91,7 +91,7 @@ if ($revision > 0) { push @$props, ['svn:log', $comment]; - push @$props, ['svn:author', $author]; + push @$props, ['svn:author', lc($author)]; } push @$props, ['svn:date', $self->svn_timestamp($timestamp)]; From toby at etjohnson.us Sat Jan 6 18:20:01 2007 From: toby at etjohnson.us (Toby Johnson) Date: Sat Jan 6 19:20:28 2007 Subject: Mixed encodings in dump file In-Reply-To: <45A03668.8000403@bezeqint.net> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> Message-ID: <45A03CB1.9040600@etjohnson.us> Ori Avtalion wrote: > Toby Johnson wrote: > >> Gotcha. I'll close ticket 26 and open another to add an encoding switch >> to ssphys. >> >> > > I have written a patch for > > > Since I have technical problems at work with running vss2svn from the > source, I cannot test it with my VSS repository. > > I know this isn't a healthy way to submit patches, but I think it will > at least help speed the ticket along, and since I made changes to ssphys > it can basically be tested with a regular latin-codepage VSS repository. > > There is a chance it may not run at all, though :) > > I've added an encoding flag to the xml formatter of ssphys with a > default value of windows-1252, and a matching flag for vss2svn.pl (also > with a default). > Thanks for the patch! It looks good to me at first glance. I currently don't have a build environment set up for ssphys, so hopefully Dirk or Kenneth can test it out before we apply it. Otherwise I'll add it to my growing to-do list... > Regarding DoSsCMD() in vss2svn.pl, the character range removed from the > output is correct for *most* windows codepages, since all of them > re-define just the lower part of windows-1255. > Arabic, for example, has rarely used characters in 0x8D and 0x90 > > Here is the line you are referring to: $gSysOut =~ s/[\x00-\x09\x11\x12\x14-\x1F\x81\x8D\x8F\x90\x9D]/_/g; Maybe we should remove the 0x81 and everything after if it's not windows-1252? Or maybe we should be more strict when it comes to filenames and less so for everything else? Are those upper codepage characters considered valid filename characters in those codepages? toby From oavtal at bezeqint.net Sun Jan 7 11:45:39 2007 From: oavtal at bezeqint.net (Ori Avtalion) Date: Sun Jan 7 12:46:31 2007 Subject: Usernames and case-sensitivity In-Reply-To: <45A038CB.60909@etjohnson.us> References: <45A025D3.9040204@bezeqint.net> <45A038CB.60909@etjohnson.us> Message-ID: <45A131C3.6070608@bezeqint.net> Toby Johnson wrote: > Ori Avtalion wrote: >> Hi, >> >> I noticed users in VSS are case-insensitive and stored as upper-case (I >> tried renaming them to lower-case without success). Is this true? >> > > They are not stored as all-uppercase in VSS, although if you initially > named them that way, VSS may not consider it a rename if you change only > the case. Perhaps it would work if you changed the name to something > else, then back to the desired case. VSS seems to force first letter to be upper case, and the others to lowercase. Thanks for the pointer to the relevant code. From toby at etjohnson.us Sun Jan 7 13:01:47 2007 From: toby at etjohnson.us (Toby Johnson) Date: Sun Jan 7 14:01:50 2007 Subject: Usernames and case-sensitivity In-Reply-To: <45A131C3.6070608@bezeqint.net> References: <45A025D3.9040204@bezeqint.net> <45A038CB.60909@etjohnson.us> <45A131C3.6070608@bezeqint.net> Message-ID: <45A1439B.2020405@etjohnson.us> Ori Avtalion wrote: > Toby Johnson wrote: > >> Ori Avtalion wrote: >> >>> Hi, >>> >>> I noticed users in VSS are case-insensitive and stored as upper-case (I >>> tried renaming them to lower-case without success). Is this true? >>> >>> >> They are not stored as all-uppercase in VSS, although if you initially >> named them that way, VSS may not consider it a rename if you change only >> the case. Perhaps it would work if you changed the name to something >> else, then back to the desired case. >> > > VSS seems to force first letter to be upper case, and the others to > lowercase. > It must be the GUI that forces that restriction. You may have luck with the following approach: * Move or rename the file um.dat in your VSS "data" directory. * Edit the "users.txt" file in the root of you VSS directory to change the names and/or case of your users. * Run VSS analyze.exe with the "-f" (fix) option on your "data" directory to restore the um.dat file with your new names. * Open VSS Admin and VSS Explorer using the new um.dat; if it doesn't work restore the old one. toby From vss2svn at nogga.de Sun Jan 7 14:08:38 2007 From: vss2svn at nogga.de (Dirk) Date: Sun Jan 7 15:08:44 2007 Subject: Mixed encodings in dump file In-Reply-To: <45A03668.8000403@bezeqint.net> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> Message-ID: <45A15346.5090309@nogga.de> > I have written a patch for > > > Tanks for the patch. I will test it tomorrow, when I'm back at my computer. I had a short look at it and it seems to be perfect. Thanks again. Dirk From stephen.lee at hexagonmetrology.com Mon Jan 8 12:54:19 2007 From: stephen.lee at hexagonmetrology.com (Stephen Lee) Date: Mon Jan 8 13:54:31 2007 Subject: Problems with import -- vss2svn In-Reply-To: References: Message-ID: <45A2935B.9040408@wilcoxassoc.com> Robert Kluver wrote: > > I saw your post in the vss2svn users mailing list. I'm tryng to > convert VSS repository to SVN and I get the same error as you. There > are probably several destroyed files in our VSS repository along with > shared files. I assume by now you have fully switched over to SVN. Did > you manage to tweak the import or did you end up just moving the files > over to a new svn repository (without history)? > No, I did the dummy run, and concluded that as it was non-trivial I would need to dedicate some more time to it, and do this after the new year (i.e. will be trying again shortly). I'll check again performance with any updated version (think I saw a message on the list recently about a new one getting created), and if I still get the same issues will go through and manually determine the correct location for each of the incorrectly orphaned files/projects (probably in the region of 50 to 200, hard to tell without doing it as there are correct orphans too), and use this for a search/replace filter program to run over the dump file to change the paths as appropriate. Branches seemed to work as best as I could determine, but unpinned and repinned files were often wrong. The way these are being used they can easily be overwritten with the correctly pinned version from sourcesafe after import, but incorrectly orphaned files/projects need to be fixed before import so that "label" branches work correctly. I'll be doing another dummy run in the next day or two, simply to answer the question of whether to do the conversion before or after attempting to merge a development branch onto the trunk (if the conversion had gone better the first time, we may well have converted to Subversion before this branch was created....) Cc-ing to list in case anyone has better suggestions for the "false orphan" problem that sometimes occurs after files/projects have been recreated in the location of a deleted file/project that previously used the same name. -- Stephen Lee Software Engineer, Vision Group - Pro-Measure Leader Wilcox Associates Inc. (U.K.) From rkluver at hotmail.com Tue Jan 9 20:27:44 2007 From: rkluver at hotmail.com (Robert Kluver) Date: Tue Jan 9 21:27:53 2007 Subject: Problems with import -- vss2svn In-Reply-To: <45A2935B.9040408@wilcoxassoc.com> Message-ID: Stephen, I also tried to run the dumpfile through "svndumpfilter" in order to extract individual projects, however it failed: type dumpfile.txt | svndumpfilter include "Data Logger" > dl-dumpfile ... Revision 5496 committed as 5496. Revision 5497 committed as 5497. Revision 5498 committed as 5498. Revision 5499 committed as 5499. svndumpfilter: Invalid copy source path '/EMx Data Logger/Source/confirm_changes_dialog.ui' The process tried to write to a nonexistent pipe. Not sure yet why this fails. Maybe the cmd on XP is not as robust as csh on linux. An option could be to write a script to remove entries from the dumpfile myself, looks feasable as long as one makes sure the revision numbers are without gaps. Did you find much information about how one can edit the dump file, or is it only from your own research? Regards, Rob > >Robert Kluver wrote: >> >>I saw your post in the vss2svn users mailing list. I'm tryng to convert >>VSS repository to SVN and I get the same error as you. There are probably >>several destroyed files in our VSS repository along with shared files. I >>assume by now you have fully switched over to SVN. Did you manage to tweak >>the import or did you end up just moving the files over to a new svn >>repository (without history)? >> >No, I did the dummy run, and concluded that as it was non-trivial I would >need to dedicate some more time to it, and do this after the new year (i.e. >will be trying again shortly). > >I'll check again performance with any updated version (think I saw a >message on the list recently about a new one getting created), and if I >still get the same issues will go through and manually determine the >correct location for each of the incorrectly orphaned files/projects >(probably in the region of 50 to 200, hard to tell without doing it as >there are correct orphans too), and use this for a search/replace filter >program to run over the dump file to change the paths as appropriate. > >Branches seemed to work as best as I could determine, but unpinned and >repinned files were often wrong. The way these are being used they can >easily be overwritten with the correctly pinned version from sourcesafe >after import, but incorrectly orphaned files/projects need to be fixed >before import so that "label" branches work correctly. > >I'll be doing another dummy run in the next day or two, simply to answer >the question of whether to do the conversion before or after attempting to >merge a development branch onto the trunk (if the conversion had gone >better the first time, we may well have converted to Subversion before this >branch was created....) > >Cc-ing to list in case anyone has better suggestions for the "false orphan" >problem that sometimes occurs after files/projects have been recreated in >the location of a deleted file/project that previously used the same name. > >-- >Stephen Lee >Software Engineer, Vision Group - Pro-Measure Leader >Wilcox Associates Inc. (U.K.) > >_______________________________________________ >vss2svn-users mailing list >Project homepage: >http://www.pumacode.org/projects/vss2svn/ >Subscribe/Unsubscribe/Admin: >http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org >Mailing list web interface (with searchable archives): >http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > _________________________________________________________________ Type your favorite song.  Get a customized station.  Try MSN Radio powered by Pandora. http://radio.msn.com/?icid=T002MSN03A07001 From wamii1 at ameritech.net Wed Jan 10 22:52:51 2007 From: wamii1 at ameritech.net (William A. Miller II) Date: Wed Jan 10 23:53:03 2007 Subject: All conversions stop at TASK: MERGEPARENTDATA Message-ID: <001a01c7353c$5a07d0c0$4001a8c0@WAMIICOMP> Any suggestions would be greatly appreciated. I have attempted to convert several databases and each stops at TASK: MERGEPARENTDATA without generating a dumpfile. I am using the 0.11.0-alpha1 build of vss2svn with both vss2svn.exe and ssphys.exe available via PATH. The error message: "DBD::SQLite2::st execute failed: unable to open database file(1) at dbdimp.c line 419 at vss2svn.pl line 590." Always occurs twice in the output of the as shown below. I am using a commandline of: vss2svn --vssdir D:/VSS6/TPDev --dumpfile d:/TPDev-dumpfile.txt --verbose --debug --timing %2 >>v2s%1.txt 2>&1 The last few lines of the output are as follows: physdir: "D:/VSS6/TPDev/data", physfolder: "Z" physname: "ZYMAAAAA" "ssphys" info "D:/VSS6/TPDev/data/Z/ZYMAAAAA" DBD::SQLite2::st execute failed: unable to open database file(1) at dbdimp.c line 419 at vss2svn.pl line 590. DBD::SQLite2::st execute failed: unable to open database file(1) at dbdimp.c line 419 at vss2svn.pl line 590. TrackingSet.cpp 1 .A D:/VSS6/TPDev/data/Z/ZYMAAAAA.A 2 0 1 0 0 0 0 0 0x0 OYMAAAAA 0 TrackingSet.cpp ZYMAAAAA 1165916577 Wamii 1 COMMITTING CACHE 'PhysicalAction' TO DATABASE SETTING TASK MERGEPARENTDATA SETTING STEP 0 TASK: MERGEPARENTDATA -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070110/5c099bbb/attachment.html From Gert.Kello at mail.ee Thu Jan 11 01:58:53 2007 From: Gert.Kello at mail.ee (Gert Kello) Date: Thu Jan 11 02:59:35 2007 Subject: All conversions stop at TASK: MERGEPARENTDATA In-Reply-To: <001a01c7353c$5a07d0c0$4001a8c0@WAMIICOMP> References: <001a01c7353c$5a07d0c0$4001a8c0@WAMIICOMP> Message-ID: <45A5EE3D.9060108@mail.ee> If You have TortoiseSVN installed, try to disable the status cache temporary (in TortoiseSVN settings, Icon overlays, status cache group) William A. Miller II wrote: > > > Any suggestions would be greatly appreciated. > > I have attempted to convert several databases and each stops at TASK: > MERGEPARENTDATA without generating a dumpfile. > > I am using the 0.11.0-alpha1 build of vss2svn with both vss2svn.exe and > ssphys.exe available via PATH. > > The error message: ?DBD::SQLite2::st execute failed: unable to open > database file(1) at dbdimp.c line 419 at vss2svn.pl line 590.? Always > occurs twice in the output of the as shown below. > > I am using a commandline of: > > vss2svn --vssdir D:/VSS6/TPDev --dumpfile d:/TPDev-dumpfile.txt > --verbose --debug --timing %2 >>v2s%1.txt 2>&1 > -- Gert -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3279 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070111/51179382/smime.bin From toby at etjohnson.us Thu Jan 11 08:56:02 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu Jan 11 09:56:06 2007 Subject: All conversions stop at TASK: MERGEPARENTDATA In-Reply-To: <45A5EE3D.9060108@mail.ee> References: <001a01c7353c$5a07d0c0$4001a8c0@WAMIICOMP> <45A5EE3D.9060108@mail.ee> Message-ID: <45A65002.4050907@etjohnson.us> Gert Kello wrote: > If You have TortoiseSVN installed, try to disable the status cache > temporary (in TortoiseSVN settings, Icon overlays, status cache group) Thanks Gert, that sounds like the voice of experience. :) I seem to recall this coming up before but couldn't remember what the resolution was; did you have the same problem? Perhaps we should list "disable TSVN" as one of the pre-conversion steps on the wiki. toby From Gert.Kello at mail.ee Thu Jan 11 09:13:18 2007 From: Gert.Kello at mail.ee (Gert Kello) Date: Thu Jan 11 10:13:27 2007 Subject: All conversions stop at TASK: MERGEPARENTDATA In-Reply-To: <45A65002.4050907@etjohnson.us> References: <001a01c7353c$5a07d0c0$4001a8c0@WAMIICOMP> <45A5EE3D.9060108@mail.ee> <45A65002.4050907@etjohnson.us> Message-ID: <45A6540E.30805@mail.ee> >> If You have TortoiseSVN installed, try to disable the status cache >> temporary (in TortoiseSVN settings, Icon overlays, status cache group) > > Thanks Gert, that sounds like the voice of experience. :) I seem to > recall this coming up before but couldn't remember what the resolution > was; did you have the same problem? Yes I had. Check my emails from 4-5. oct. Standard error redirection was required to reproduce the error... > Perhaps we should list "disable > TSVN" as one of the pre-conversion steps on the wiki. :D -- Gert -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3279 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070111/a7f69443/smime-0001.bin From wamii1 at ameritech.net Thu Jan 11 10:57:20 2007 From: wamii1 at ameritech.net (William A. Miller II) Date: Thu Jan 11 11:57:29 2007 Subject: All conversions stop at TASK: MERGEPARENTDATA In-Reply-To: <45A6540E.30805@mail.ee> Message-ID: <006601c735a1$8f3aa320$4001a8c0@WAMIICOMP> That fixed the problem. Thanks. I agree that there should be a reference to this fix in the wiki. I would never have associated a TortoiseSVN setting as causing the problem without your help. -----Original Message----- From: vss2svn-users-bounces@lists.pumacode.org [mailto:vss2svn-users-bounces@lists.pumacode.org] On Behalf Of Gert Kello Sent: Thursday, January 11, 2007 9:13 AM To: Vss2Svn Users Subject: Re: All conversions stop at TASK: MERGEPARENTDATA >> If You have TortoiseSVN installed, try to disable the status cache >> temporary (in TortoiseSVN settings, Icon overlays, status cache group) > > Thanks Gert, that sounds like the voice of experience. :) I seem to > recall this coming up before but couldn't remember what the resolution > was; did you have the same problem? Yes I had. Check my emails from 4-5. oct. Standard error redirection was required to reproduce the error... > Perhaps we should list "disable > TSVN" as one of the pre-conversion steps on the wiki. :D -- Gert From toby at etjohnson.us Thu Jan 11 11:44:47 2007 From: toby at etjohnson.us (Toby Johnson) Date: Thu Jan 11 12:44:51 2007 Subject: All conversions stop at TASK: MERGEPARENTDATA In-Reply-To: <006601c735a1$8f3aa320$4001a8c0@WAMIICOMP> References: <006601c735a1$8f3aa320$4001a8c0@WAMIICOMP> Message-ID: <45A6778F.4000105@etjohnson.us> William A. Miller II wrote: > That fixed the problem. Thanks. > I agree that there should be a reference to this fix in the wiki. > I would never have associated a TortoiseSVN setting as causing the problem > without your help. > Glad it worked for you. I've added "Disable TortoiseSVN" as a step in the Running the Migration wiki. From jtavares at spsy.com Fri Jan 12 07:50:01 2007 From: jtavares at spsy.com (James Tavares) Date: Fri Jan 12 08:50:11 2007 Subject: Problems with import -- vss2svn Message-ID: <09D1B03EA3A4EC46AA5B2A0697C6C4536096E5@xmpp.spsy.com> Rob: I'm guessing that your /Data Logger directory was at one point copied from /EMx Data Logger/... Svndumpfilter is a very "stupid" program. It does not follow follow copies. Try this: type dumpfile.txt | svndumpfilter include "Data Logger" "EMx Data Logger" \ > dl-dumpfile The idea is you have to supply svndumpfilter not only with the path you want to copy, but also all of the paths that files in your primary path could have been copied from. It's a crappy setup -- someone should extend svndumpfilter with options to automatically follow copies. Also, if you include a "deep" path (more than one level) -- svndumpfilter will not create the root path(s) for you. You have to make entries in the dump file yourself, or create them in your target repository before doing the import. -James > -----Original Message----- > From: vss2svn-users-bounces@lists.pumacode.org [mailto:vss2svn-users- > bounces@lists.pumacode.org] On Behalf Of Robert Kluver > Sent: Tuesday, January 09, 2007 9:28 PM > To: vss2svn-users@lists.pumacode.org > Subject: Re: Problems with import -- vss2svn > > Stephen, > > I also tried to run the dumpfile through "svndumpfilter" in order to > extract > individual projects, however it failed: > > > type dumpfile.txt | svndumpfilter include "Data Logger" > dl-dumpfile > > ... > Revision 5496 committed as 5496. > Revision 5497 committed as 5497. > Revision 5498 committed as 5498. > Revision 5499 committed as 5499. > svndumpfilter: Invalid copy source path '/EMx Data > Logger/Source/confirm_changes_dialog.ui' > The process tried to write to a nonexistent pipe. > > Not sure yet why this fails. Maybe the cmd on XP is not as robust as csh > on > linux. > > An option could be to write a script to remove entries from the dumpfile > myself, looks feasable as long as one makes sure the revision numbers are > without gaps. > > Did you find much information about how one can edit the dump file, or is > it > only from your own research? > > Regards, > > Rob > > > > > > >Robert Kluver wrote: > >> > >>I saw your post in the vss2svn users mailing list. I'm tryng to convert > >>VSS repository to SVN and I get the same error as you. There are > probably > >>several destroyed files in our VSS repository along with shared files. I > >>assume by now you have fully switched over to SVN. Did you manage to > tweak > >>the import or did you end up just moving the files over to a new svn > >>repository (without history)? > >> > >No, I did the dummy run, and concluded that as it was non-trivial I would > >need to dedicate some more time to it, and do this after the new year > (i.e. > >will be trying again shortly). > > > >I'll check again performance with any updated version (think I saw a > >message on the list recently about a new one getting created), and if I > >still get the same issues will go through and manually determine the > >correct location for each of the incorrectly orphaned files/projects > >(probably in the region of 50 to 200, hard to tell without doing it as > >there are correct orphans too), and use this for a search/replace filter > >program to run over the dump file to change the paths as appropriate. > > > >Branches seemed to work as best as I could determine, but unpinned and > >repinned files were often wrong. The way these are being used they can > >easily be overwritten with the correctly pinned version from sourcesafe > >after import, but incorrectly orphaned files/projects need to be fixed > >before import so that "label" branches work correctly. > > > >I'll be doing another dummy run in the next day or two, simply to answer > >the question of whether to do the conversion before or after attempting > to > >merge a development branch onto the trunk (if the conversion had gone > >better the first time, we may well have converted to Subversion before > this > >branch was created....) > > > >Cc-ing to list in case anyone has better suggestions for the "false > orphan" > >problem that sometimes occurs after files/projects have been recreated in > >the location of a deleted file/project that previously used the same > name. > > > >-- > >Stephen Lee > >Software Engineer, Vision Group - Pro-Measure Leader > >Wilcox Associates Inc. (U.K.) > > > >_______________________________________________ > >vss2svn-users mailing list > >Project homepage: > >http://www.pumacode.org/projects/vss2svn/ > >Subscribe/Unsubscribe/Admin: > >http://lists.pumacode.org/mailman/listinfo/vss2svn-users- > lists.pumacode.org > >Mailing list web interface (with searchable archives): > >http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > > _________________________________________________________________ > Type your favorite song.? Get a customized station.? Try MSN Radio powered > by Pandora. http://radio.msn.com/?icid=T002MSN03A07001 > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users- > lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user From Tom.Widmer at camcog.com Fri Jan 12 11:29:11 2007 From: Tom.Widmer at camcog.com (Tom Widmer) Date: Fri Jan 12 12:29:18 2007 Subject: VSS top level labels Message-ID: <120501C00026094E952D14B20F0C09360F39E9@MAIL.int.camcog.com> I've been trying to use vss2svn to migrate our VSS repository to SVN. However, our repository has over 1000 top level labels (of $\) which are added by our automated build process. As indicated in the thread about "Label support", you are aware that this leads to over 2000 levels of nesting of the labels folder. E.g. Labels ... -build.1000 -labels ... -build.999 -labels ... -build.998 ... (2000 levels of nesting) The obvious solution to this is to (optionally) place the repository in a subfolder. E.g. labels -build.1 Trunk-contents ... -build.1000 trunk-contents trunk trunk-contents Where $\ maps to trunk\. I've had a look at the scripts in the hope of making the modification myself (and of course passing it back), but I'm not really familiar enough with perl or the vss2svn scripts to work out what needs changing. Could you either give me an outline of what I need to change, or take this as an enhancement request? (I imagine there are various stages at which the "trunk" bit could be inserted, but someone else probably has a feeling for the best stage to add it. Would it work to do a simple minded thing in the Node module, adding "trunk" to the start of the paths if and only if they don't begin with "labels"?) Thanks, Tom ********************************************************************** IMPORTANT NOTICE. Confidentiality: This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must take no action based on them, nor must you copy or show them to anyone; please reply to this e-mail and highlight the error. Security Warning: Please note that this e-mail has been created in the knowledge that Internet e-mail is not a 100% secure communications medium. We advise that you understand and observe this lack of security when e-mailing us. Viruses: Although we have taken steps to ensure that this e-mail and attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free. Monitoring and Scanning: Cambridge Cognition has monitoring and scanning systems in place in relation to emails sent and received to: monitor / record business communications; prevent and detect crime; investigate the use of the Company's internal and external email system; and provide evidence of compliance with business practices. ********************************************************************** From toby at etjohnson.us Mon Jan 15 12:04:54 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jan 15 13:04:57 2007 Subject: Mailing list errors Message-ID: <45ABC246.60001@etjohnson.us> Due to a configuration error following a software upgrade, some emails addressed to this list in the past 24 hours may have bounced back as undeliverable. This problem has now been fixed; please feel free to resend any emails which were returned. Thanks, toby From toby at etjohnson.us Wed Jan 17 17:02:07 2007 From: toby at etjohnson.us (Toby Johnson) Date: Wed Jan 17 18:02:11 2007 Subject: Mixed encodings in dump file In-Reply-To: <45A15346.5090309@nogga.de> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> Message-ID: <45AEAAEF.5040607@etjohnson.us> Dirk wrote: > >> I have written a patch for >> >> >> > Tanks for the patch. I will test it tomorrow, when I'm back at my > computer. I had a short look at it and it seems to be perfect. Thanks > again. Dirk, did you ever get a chance to test this patch? Would it be possible for you to upload a new ssphys.exe? Send me a private email if you need the login details again. From toby at etjohnson.us Wed Jan 17 17:39:02 2007 From: toby at etjohnson.us (Toby Johnson) Date: Wed Jan 17 18:39:06 2007 Subject: Merged restore-widetime branch Message-ID: <45AEB396.1010001@etjohnson.us> I have merged Ken's restore-widetime branch back to trunk, and deleted the branch. Dirk, if you do get a chance to build ssphys for us, please be sure to use the latest trunk (r273). Hopefully then I'll be able to get out a 0.11.0-alpha2 zipfile. toby From vss2svn at nogga.de Thu Jan 18 17:10:01 2007 From: vss2svn at nogga.de (Dirk) Date: Thu Jan 18 18:10:12 2007 Subject: VSS top level labels In-Reply-To: <120501C00026094E952D14B20F0C09360F39E9@MAIL.int.camcog.com> References: <120501C00026094E952D14B20F0C09360F39E9@MAIL.int.camcog.com> Message-ID: <45AFFE49.9040008@nogga.de> Hello Tom, I have the same problem, and I was looking for a fix a long time also. Possibly there is an easy fix, if you look here (1). This is the full path name lookup code. It will recurse down all parents until we reach the root node and then we assign the root folder name "/". Possibly it is sufficient to set this to "/trunk/". Later (2), when the Labels are handled, we have a little problem. Now the label directory is /Labels/ but all pathes are encoded as /trunk/... So we will end up with /Labels/label/trunk/... Not perfect either, but a possible solution... We could naturally remove the "label base" from the path, with the "label base" beeing the directory where we assigned the label. But this would brake "Label promotion". How would we handle a situation, when a label is assigned to some path. Later the same label is assigned to a file within the path. In the current situation the labels directory is a hierarchical mirror of the trunk. It is not depending on the place where you assigned the label. It will always mirror the complete structure. I will give it a try. Best regards Dirk [1] http://www.pumacode.org/projects/vss2svn/browser/trunk/script/Vss2Svn/ActionHandler.pm#L717 [2] http://www.pumacode.org/projects/vss2svn/browser/trunk/script/Vss2Svn/Dumpfile.pm#L548 Tom Widmer schrieb: > I've been trying to use vss2svn to migrate our VSS repository to SVN. > However, our repository has over 1000 top level labels (of $\) which are > added by our automated build process. As indicated in the thread about > "Label support", you are aware that this leads to over 2000 levels of > nesting of the labels folder. E.g. > > Labels > ... > -build.1000 > -labels > ... > -build.999 > -labels > ... > -build.998 > ... (2000 levels of nesting) > > The obvious solution to this is to (optionally) place the repository in > a subfolder. E.g. > > labels > -build.1 > Trunk-contents > ... > -build.1000 > trunk-contents > trunk > trunk-contents > > Where $\ maps to trunk\. > > I've had a look at the scripts in the hope of making the modification > myself (and of course passing it back), but I'm not really familiar > enough with perl or the vss2svn scripts to work out what needs changing. > Could you either give me an outline of what I need to change, or take > this as an enhancement request? > > (I imagine there are various stages at which the "trunk" bit could be > inserted, but someone else probably has a feeling for the best stage to > add it. Would it work to do a simple minded thing in the Node module, > adding "trunk" to the start of the paths if and only if they don't begin > with "labels"?) > > Thanks, > > Tom > ********************************************************************** > IMPORTANT NOTICE. > Confidentiality: This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must take no action based on them, nor must you copy or show them to anyone; please reply to this e-mail and highlight the error. > Security Warning: Please note that this e-mail has been created in the knowledge that Internet e-mail is not a 100% secure communications medium. > We advise that you understand and observe this lack of security when e-mailing us. > Viruses: Although we have taken steps to ensure that this e-mail and attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free. > Monitoring and Scanning: Cambridge Cognition has monitoring and scanning systems in place in relation to emails sent and received to: monitor / record business communications; prevent and detect crime; investigate the use of the Company's internal and external email system; and provide evidence of compliance with business practices. > ********************************************************************** > > > > _______________________________________________ > vss2svn-users mailing list > Project homepage: > http://www.pumacode.org/projects/vss2svn/ > Subscribe/Unsubscribe/Admin: > http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists.pumacode.org > Mailing list web interface (with searchable archives): > http://dir.gmane.org/gmane.comp.version-control.subversion.vss2svn.user > > > > From eli.carter at commprove.com Fri Jan 19 15:42:10 2007 From: eli.carter at commprove.com (Eli Carter) Date: Fri Jan 19 16:42:35 2007 Subject: uninitialized values Message-ID: <200701191542.10866.eli.carter@commprove.com> Using a checkout of trunk from today, running on Linux (FC6): Connecting to database ../conversion-20070119h/my_vss-temp/vss_data.db Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line 1144. Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line 1144. Use of uninitialized value in numeric eq (==) at ./vss2svn.pl line 954. Use of uninitialized value in numeric lt (<) at ./vss2svn.pl line 954. ERROR -- The conversion needs at least ssphys version 0.22 at ./vss2svn.pl line 954 ======== VSS2SVN ======== BEGINNING CONVERSION... Start Time : Fri Jan 19 21:28:38 2007 VSS Dir : ../conversion-20070119h/DBSourceSafe Temp Dir : ../conversion-20070119h/my_vss-temp Dumpfile : ../conversion-20070119h/my_vss.raw.dump VSS Encoding : windows-1252 SSPHYS exe : ./ssphys SSPHYS ver : . XML Parser : XML::Simple From eli.carter at commprove.com Fri Jan 19 16:37:04 2007 From: eli.carter at commprove.com (Eli Carter) Date: Fri Jan 19 17:37:17 2007 Subject: [Patch] Build.PL missing SanityChecker.pm Message-ID: <200701191637.05362.eli.carter@commprove.com> The install seems to miss SanityChecker.pm. Patch attached. Eli -------------- next part -------------- A non-text attachment was scrubbed... Name: build-fix.patch Type: text/x-diff Size: 628 bytes Desc: not available Url : http://lists.pumacode.org/pipermail/vss2svn-users-lists.pumacode.org/attachments/20070119/4c4c664d/build-fix.bin From toby at etjohnson.us Fri Jan 19 17:35:20 2007 From: toby at etjohnson.us (Toby Johnson) Date: Fri Jan 19 18:35:23 2007 Subject: uninitialized values In-Reply-To: <200701191542.10866.eli.carter@commprove.com> References: <200701191542.10866.eli.carter@commprove.com> Message-ID: <45B155B8.9060008@etjohnson.us> Eli Carter wrote: > Using a checkout of trunk from today, running on Linux (FC6): > > Connecting to database ../conversion-20070119h/my_vss-temp/vss_data.db > > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > 1144. > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > 1144. > Use of uninitialized value in numeric eq (==) at ./vss2svn.pl line 954. > Use of uninitialized value in numeric lt (<) at ./vss2svn.pl line 954. > ERROR -- The conversion needs at least ssphys version 0.22 > at ./vss2svn.pl line 954 > "ERROR -- The conversion needs at least ssphys version 0.22" is the operative line here! I don't think Dirk has yet had a chance to upload it, so if you're using trunk you need to compile ssphys from source for the time being. toby From toby at etjohnson.us Fri Jan 19 17:41:07 2007 From: toby at etjohnson.us (Toby Johnson) Date: Fri Jan 19 18:41:11 2007 Subject: [Patch] Build.PL missing SanityChecker.pm In-Reply-To: <200701191637.05362.eli.carter@commprove.com> References: <200701191637.05362.eli.carter@commprove.com> Message-ID: <45B15713.70708@etjohnson.us> Eli Carter wrote: > The install seems to miss SanityChecker.pm. > > Patch attached. > > Eli > > ------------------------------------------------------------------------ > > Index: vss2svn-trunk.svn/Build.PL > =================================================================== > --- vss2svn-trunk.svn/Build.PL (revision 275) > +++ vss2svn-trunk.svn/Build.PL (working copy) > @@ -20,6 +20,7 @@ > 'script/Vss2Svn/Dumpfile.pm' => 'lib/Vss2Svn/Dumpfile.pm', > 'script/Vss2Svn/SvnRevHandler.pm' => 'lib/Vss2Svn/SvnRevHandler.pm', > 'script/Vss2Svn/Dumpfile/Node.pm' => 'lib/Vss2Svn/Dumpfile/Node.pm', > + 'script/Vss2Svn/Dumpfile/SanityChecker.pm' => 'lib/Vss2Svn/Dumpfile/SanityChecker.pm', > }, > PL_files => { 'ssphys/ssphys.PL' => 'bin/ssphys' }, > )->create_build_script; > Thanks, I've applied this in r277. From vss2svn at nogga.de Fri Jan 19 17:46:13 2007 From: vss2svn at nogga.de (Dirk) Date: Fri Jan 19 18:46:20 2007 Subject: uninitialized values In-Reply-To: <200701191542.10866.eli.carter@commprove.com> References: <200701191542.10866.eli.carter@commprove.com> Message-ID: <45B15845.6070500@nogga.de> > Connecting to database ../conversion-20070119h/my_vss-temp/vss_data.db > > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > 1144. > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > 1144. > Use of uninitialized value in numeric eq (==) at ./vss2svn.pl line 954. > Use of uninitialized value in numeric lt (<) at ./vss2svn.pl line 954. > ERROR -- The conversion needs at least ssphys version 0.22 > Sorry for that, I wanted to upload the current ssphys also but failed yesterday evening. There was a change in the command line interface that needed a newer version of ssphys. Please download the new version from: http://www.pumacode.org/download/vss2svn/ssphys/ssphys-0.22.0.zip Dirk From vss2svn at nogga.de Fri Jan 19 17:51:50 2007 From: vss2svn at nogga.de (Dirk) Date: Fri Jan 19 18:51:57 2007 Subject: [Patch] Build.PL missing SanityChecker.pm In-Reply-To: <45B15713.70708@etjohnson.us> References: <200701191637.05362.eli.carter@commprove.com> <45B15713.70708@etjohnson.us> Message-ID: <45B15996.6030509@nogga.de> > > Thanks, I've applied this in r277. You have been 2 minutes faster than me ;-) I have uploaded ssphys and also made some more changes to be able to have a different conversion root. works quite nicely here. Dirk From vss2svn at nogga.de Fri Jan 19 18:04:18 2007 From: vss2svn at nogga.de (Dirk) Date: Fri Jan 19 19:04:25 2007 Subject: uninitialized values In-Reply-To: <200701191542.10866.eli.carter@commprove.com> References: <200701191542.10866.eli.carter@commprove.com> Message-ID: <45B15C82.6010309@nogga.de> > XML Parser : XML::Simple > Just to let you know: You can speed up the conversion process if you install the SAX::Expat parser. Additionally you need to modify the Perl site ParserDetails.ini file (C:\Programme\Perl\site\lib\XML\SAX\ParserDetails.ini) and set it to the following settings: [XML::SAX::Expat] http://xml.org/sax/features/namespaces = 1 http://xml.org/sax/features/validation = 1 Best regards Dirk From toby at etjohnson.us Sat Jan 20 13:07:31 2007 From: toby at etjohnson.us (Toby Johnson) Date: Sat Jan 20 14:07:37 2007 Subject: [Patch] Build.PL missing SanityChecker.pm In-Reply-To: <45B15996.6030509@nogga.de> References: <200701191637.05362.eli.carter@commprove.com> <45B15713.70708@etjohnson.us> <45B15996.6030509@nogga.de> Message-ID: <45B26873.902@etjohnson.us> Dirk wrote: > >> >> Thanks, I've applied this in r277. > > You have been 2 minutes faster than me ;-) > > I have uploaded ssphys and also made some more changes to be able to > have a different conversion root. works quite nicely here. Great! I'll try to put together a new zipfile with and .exe for the script sometime this week. toby From vss2svn at nogga.de Sun Jan 21 07:58:01 2007 From: vss2svn at nogga.de (Dirk) Date: Sun Jan 21 08:58:08 2007 Subject: [Patch] Build.PL missing SanityChecker.pm In-Reply-To: <45B26873.902@etjohnson.us> References: <200701191637.05362.eli.carter@commprove.com> <45B15713.70708@etjohnson.us> <45B15996.6030509@nogga.de> <45B26873.902@etjohnson.us> Message-ID: <45B37169.40801@nogga.de> >> You have been 2 minutes faster than me ;-) >> >> I have uploaded ssphys and also made some more changes to be able to >> have a different conversion root. works quite nicely here. > > Great! I'll try to put together a new zipfile with and .exe for the > script sometime this week. I found, that there is a unicode filename issue again. I have not yet undestood when perl is interpreting the string as "already" in utf-8. In some cases, not in all, I see a double encoding of utf-8 filenames. The problem is that all temporary database text files, e.g. Physical Action, VssAction are ok, but the final dumpfile will show double encoded entries. I'll have some time on monday evening. I will try to solve it then. Dikr From eli.carter at commprove.com Mon Jan 22 09:24:45 2007 From: eli.carter at commprove.com (Eli Carter) Date: Mon Jan 22 10:25:16 2007 Subject: uninitialized values In-Reply-To: <45B155B8.9060008@etjohnson.us> References: <200701191542.10866.eli.carter@commprove.com> <45B155B8.9060008@etjohnson.us> Message-ID: <200701220924.45820.eli.carter@commprove.com> On Friday 19 January 2007 5:35 pm, Toby Johnson wrote: > Eli Carter wrote: > > Using a checkout of trunk from today, running on Linux (FC6): > > > > Connecting to database ../conversion-20070119h/my_vss-temp/vss_data.db > > > > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > > 1144. > > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > > 1144. > > Use of uninitialized value in numeric eq (==) at ./vss2svn.pl line 954. > > Use of uninitialized value in numeric lt (<) at ./vss2svn.pl line 954. > > ERROR -- The conversion needs at least ssphys version 0.22 > > at ./vss2svn.pl line 954 > > > > "ERROR -- The conversion needs at least ssphys version 0.22" is the > operative line here! I don't think Dirk has yet had a chance to upload > it, so if you're using trunk you need to compile ssphys from source for > the time being. I'm building ssphys from the same svn checkout as the vss2svn. Is the ssphys source in http://svn.pumacode.org/repos/vss2svn/trunk/ssphys not the right ssphys for the vss2svn in http://svn.pumacode.org/repos/vss2svn/trunk/script ??? Eli From eli.carter at commprove.com Mon Jan 22 09:28:48 2007 From: eli.carter at commprove.com (Eli Carter) Date: Mon Jan 22 10:29:07 2007 Subject: uninitialized values In-Reply-To: <45B15845.6070500@nogga.de> References: <200701191542.10866.eli.carter@commprove.com> <45B15845.6070500@nogga.de> Message-ID: <200701220928.49448.eli.carter@commprove.com> On Friday 19 January 2007 5:46 pm, Dirk wrote: > > > Connecting to database ../conversion-20070119h/my_vss-temp/vss_data.db > > > > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > > 1144. > > Use of uninitialized value in concatenation (.) or string at ./vss2svn.pl line > > 1144. > > Use of uninitialized value in numeric eq (==) at ./vss2svn.pl line 954. > > Use of uninitialized value in numeric lt (<) at ./vss2svn.pl line 954. > > ERROR -- The conversion needs at least ssphys version 0.22 > > > > Sorry for that, I wanted to upload the current ssphys also but failed > yesterday evening. There was a change in the command line interface that > needed a newer version of ssphys. > > Please download the new version from: > http://www.pumacode.org/download/vss2svn/ssphys/ssphys-0.22.0.zip Thanks, but an .exe isn't very helpful on this Linux box. Can you please commit the source to http://svn.pumacode.org/repos/vss2svn/trunk/ssphys so that it's up-to-date with what the code in http://svn.pumacode.org/repos/vss2svn/trunk/script expects? Eli From vss2svn at nogga.de Mon Jan 22 09:31:20 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jan 22 10:31:16 2007 Subject: uninitialized values In-Reply-To: <200701220924.45820.eli.carter@commprove.com> References: <200701191542.10866.eli.carter@commprove.com> <45B155B8.9060008@etjohnson.us> <200701220924.45820.eli.carter@commprove.com> Message-ID: <45B4D8C8.8030909@nogga.de> > I'm building ssphys from the same svn checkout as the vss2svn. Is the ssphys > source in http://svn.pumacode.org/repos/vss2svn/trunk/ssphys not the right > ssphys for the vss2svn in > http://svn.pumacode.org/repos/vss2svn/trunk/script ??? > The current "version.in" in trunk lists 0.22.0 as the current version number for ssphys (see http://svn.pumacode.org/repos/vss2svn/trunk/ssphys/version.in). Possibly you need to force a rebuild on this file. Dirk From eli.carter at commprove.com Mon Jan 22 13:08:18 2007 From: eli.carter at commprove.com (Eli Carter) Date: Mon Jan 22 14:08:34 2007 Subject: not well-formed (invalid token) Message-ID: <200701221308.18694.eli.carter@commprove.com> I'm getting this error message: ... "./ssphys" info -ewindows-1252 "../t/myvss/data/p/peeaaaaa" WARNING: control character 0x12 in text input at character 2 WARNING: control character 0x01 in text input at character 1 WARNING: control character 0x01 in text input at character 1 WARNING: control character 0x12 in text input at character 2 not well-formed (invalid token) at line 23, column 16, byte 835 at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/XML/Parser.pm line 187 I expect that means that the input file is corrupted in some fashion? (As opposed to a typo in the XML parser.) Another possibility is that I need to specify a different unicode encoding? Any suggestions on how to address this? Eli From vss2svn at nogga.de Mon Jan 22 14:37:13 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jan 22 15:37:21 2007 Subject: not well-formed (invalid token) In-Reply-To: <200701221308.18694.eli.carter@commprove.com> References: <200701221308.18694.eli.carter@commprove.com> Message-ID: <45B52079.3030600@nogga.de> > not well-formed (invalid token) at line 23, column 16, byte 835 > at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/XML/Parser.pm > line 187 > > I expect that means that the input file is corrupted in some fashion? (As > opposed to a typo in the XML parser.) > Yes, I would expect this, too. Try to run the command standalone and have a look at the XML file. Can you spot the problematic location? There was/is some code in vss2svn to remove any invalid characters. Possibly this is wrong with your encoding, since it probably removes "valid" elements. Try to remove this code or extend it with the invalid characters. Have a special look at this line: http://www.pumacode.org/projects/vss2svn/browser/trunk/script/vss2svn.pl#L1096 This will remove invalid windows-1252 characters. Probably this is problematic. Is it possible for you to run the generated XML through a XML-Checker? While I was researching XML, I found that there are some characters forbidden in XML, which are valid characters in some codepages. Even if I encoded them in the "dezimal" encoding, they where reported as bad input. So I don't know how to deal with this. Dirk From toby at etjohnson.us Mon Jan 22 14:44:59 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jan 22 15:45:05 2007 Subject: uninitialized values In-Reply-To: <45B5210A.6020709@nogga.de> References: <200701191542.10866.eli.carter@commprove.com> <200701221024.58996.eli.carter@commprove.com> <45B4F52F.7010500@nogga.de> <200701221150.57951.eli.carter@commprove.com> <45B5210A.6020709@nogga.de> Message-ID: <45B5224B.4010400@etjohnson.us> Dirk wrote: > Eli Carter schrieb: >> On Monday 22 January 2007 11:32 am, Dirk wrote: >> >>>> Huh. I just did a re-build, and it seems to be working. Ok, chalk >>>> one uto user error. Sorry for the noise. >>>> >>> No problem. After several years of working, I came across a lot of >>> situations where we debugged like crazy, involved other people and >>> finally a rebuild solved all issues ;-) And I still step into the >>> same pitfall over and over again. >>> >> >> Second user error: testing on two different machines. >> >> Here's the real problem: >> ./ssphys --version >> ./ssphys: error while loading shared libraries: >> libboost_filesystem.so.2: cannot open shared object file: No such >> file or directory >> >> So, if ssphys fails to run, it complains about the version of ssphys, >> rather than complaining that it failed to run. A more accurate error >> message would be preferred. >> > Good point, indeed. A programmer error. I thought about adding some > code to check whether ssphys failed to start for some reason. But > since I'm not a perl expert, I failed in my attempts. So probably this > is a task for Toby ;-) OK, I'll set up an issue for this; I'm assuming this is possible by checking the return value of the system() call. Please try to copy in the mailing list on these issues if possible, thanks! toby From vss2svn at nogga.de Mon Jan 22 17:16:30 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jan 22 18:16:37 2007 Subject: Mixed encodings in dump file In-Reply-To: <45AEAAEF.5040607@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> <45AEAAEF.5040607@etjohnson.us> Message-ID: <45B545CE.2040901@nogga.de> > Dirk, did you ever get a chance to test this patch? Would it be > possible for you to upload a new ssphys.exe? Send me a private email > if you need the login details again. Ok, I have finished testing. Now all labels, filenames, comments, and so on are in the correct encoding (* at least on my testdatabase) So please try. @Toby: I think you can start to package a new version. Best regards Dirk From toby at etjohnson.us Mon Jan 22 17:54:57 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jan 22 18:55:01 2007 Subject: Mixed encodings in dump file In-Reply-To: <45B545CE.2040901@nogga.de> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> <45AEAAEF.5040607@etjohnson.us> <45B545CE.2040901@nogga.de> Message-ID: <45B54ED1.7060203@etjohnson.us> Dirk wrote: > >> Dirk, did you ever get a chance to test this patch? Would it be >> possible for you to upload a new ssphys.exe? Send me a private email >> if you need the login details again. > > Ok, I have finished testing. Now all labels, filenames, comments, and > so on are in the correct encoding (* at least on my testdatabase) So > please try. > > @Toby: I think you can start to package a new version. Sounds good... so I'm still using .exe version 0.22.0 from 19 January, correct? toby From vss2svn at nogga.de Mon Jan 22 18:10:53 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jan 22 19:10:59 2007 Subject: Mixed encodings in dump file In-Reply-To: <45B54ED1.7060203@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> <45AEAAEF.5040607@etjohnson.us> <45B545CE.2040901@nogga.de> <45B54ED1.7060203@etjohnson.us> Message-ID: <45B5528D.40906@nogga.de> >> >> @Toby: I think you can start to package a new version. > > Sounds good... so I'm still using .exe version 0.22.0 from 19 January, > correct? Yes, the issue was in vss2svn, not in ssphys. If you have time, you could probably add a better check for a wrong version number or a failed start of ssphys. I'm currently working on the wrong timestamp issue. I'm about to check in a modification for this within the next minutes. Dirk From vss2svn at nogga.de Mon Jan 22 19:57:31 2007 From: vss2svn at nogga.de (Dirk) Date: Mon Jan 22 20:57:38 2007 Subject: auto-properties Message-ID: <45B56B8B.8020601@nogga.de> I started a new module AutoProps to start development on this issue. I is currently only a simple module that will read a subversion config. But there is no functionality in there. Only a test. We will get a hash of filepatterns/properties. So the next step is to evaluate the itempath against all filepatterns and to add all properties if the filepattern matches. Discussion: * if mulitple patterns match, all properties of all matching patterns should be added * per file pattern multiple properties can be specified, seperated by semicolon * What should happen if the same property is added twice * with the same value * with a different value * where to integrate? * Dumpfile::_add_handler * Node::Set_initial_properties * can properties change in a subversion "change" node? Since I'm not the best PERL programmer I just have started the development. Anybody out there, to pick it up from here ;-) Best regards Dirk From toby at etjohnson.us Mon Jan 22 20:02:19 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jan 22 21:02:22 2007 Subject: Mixed encodings in dump file In-Reply-To: <45B5528D.40906@nogga.de> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> <45AEAAEF.5040607@etjohnson.us> <45B545CE.2040901@nogga.de> <45B54ED1.7060203@etjohnson.us> <45B5528D.40906@nogga.de> Message-ID: <45B56CAB.9020802@etjohnson.us> Dirk wrote: > >>> >>> @Toby: I think you can start to package a new version. >> >> Sounds good... so I'm still using .exe version 0.22.0 from 19 >> January, correct? > Yes, the issue was in vss2svn, not in ssphys. If you have time, you > could probably add a better check for a wrong version number or a > failed start of ssphys. > > I'm currently working on the wrong timestamp issue. I'm about to check > in a modification for this within the next minutes. Thanks Dirk, glad to see you have some free time lately :) Who knows, maybe I'll even find some time to knock out a few tickets myself! From toby at etjohnson.us Mon Jan 22 20:08:46 2007 From: toby at etjohnson.us (Toby Johnson) Date: Mon Jan 22 21:08:49 2007 Subject: auto-properties In-Reply-To: <45B56B8B.8020601@nogga.de> References: <45B56B8B.8020601@nogga.de> Message-ID: <45B56E2E.2000800@etjohnson.us> Dirk wrote: > I started a new module AutoProps to start development on this issue. I > is currently only a simple module that will read a subversion config. > But there is no functionality in there. Only a test. We will get a > hash of filepatterns/properties. So the next step is to evaluate the > itempath against all filepatterns and to add all properties if the > filepattern matches. > > Discussion: > > * if mulitple patterns match, all properties of all matching patterns > should be added > * per file pattern multiple properties can be specified, seperated by > semicolon > * What should happen if the same property is added twice > * with the same value > * with a different value I'm pretty sure I've read at one point in the Subversion docs that adding the same property multiple times causes an indeterminate result, but that the current implementation is that the "last one wins". At any rate, that sounds like a "good enough" solution to me. So that means we should build a hash of property => value pairs for each node. > * where to integrate? > * Dumpfile::_add_handler > * Node::Set_initial_properties Auto-props are only used by Subversion when a file is first added, so I think Dumpfile::_add_handler() makes sense. > * can properties change in a subversion "change" node? I'm pretty sure they can, although it's a moot point for autoprops since they're only used on adds. > Since I'm not the best PERL programmer I just have started the > development. Anybody out there, to pick it up from here ;-) Thanks, it's a good start. :) I can probably knock this out when I get some free time. toby From vss2svn at nogga.de Tue Jan 23 03:06:28 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jan 23 04:06:48 2007 Subject: Ticket triage? In-Reply-To: <45B56CAB.9020802@etjohnson.us> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> <45AEAAEF.5040607@etjohnson.us> <45B545CE.2040901@nogga.de> <45B54ED1.7060203@etjohnson.us> <45B5528D.40906@nogga.de> <45B56CAB.9020802@etjohnson.us> Message-ID: <45B5D014.5060103@nogga.de> > > Thanks Dirk, glad to see you have some free time lately :) Who knows, > maybe I'll even find some time to knock out a few tickets myself! I think a few tickets that are still open are already closed. We should have a look at the list. Dirk From vss2svn at nogga.de Tue Jan 23 03:13:16 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jan 23 04:13:34 2007 Subject: auto-properties In-Reply-To: <45B56E2E.2000800@etjohnson.us> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> Message-ID: <45B5D1AC.6050702@nogga.de> >> * where to integrate? >> * Dumpfile::_add_handler >> * Node::Set_initial_properties > > Auto-props are only used by Subversion when a file is first added, so > I think Dumpfile::_add_handler() makes sense. I thought about this a little further. As far as I understand, auto_props only work on the filename. So the _add_handler is the correct place. If we also want to filter/check the content of the file, then we must find a good place in Node.pm. I would also suggest to start with the _add_handler and its derivates like share without a source, restore, recover and so on. > > Thanks, it's a good start. :) I can probably knock this out when I get > some free time. The main problem for me is, that it takes me hours to write a simple line in PERL. Opposed to what is written in all its documentation, perl is not an easy language. It might be powerful, but not easy to read and to code. Perhaps I will play a little further. I just need to have an idea, how to evaluate the auto-props-pattern from the hash against the filename. Dirk From toby at etjohnson.us Tue Jan 23 09:27:55 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jan 23 10:27:58 2007 Subject: auto-properties In-Reply-To: <45B5D1AC.6050702@nogga.de> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> Message-ID: <45B6297B.4090304@etjohnson.us> Dirk wrote: > >>> * where to integrate? >>> * Dumpfile::_add_handler >>> * Node::Set_initial_properties >> >> Auto-props are only used by Subversion when a file is first added, so >> I think Dumpfile::_add_handler() makes sense. > > I thought about this a little further. As far as I understand, > auto_props only work on the filename. So the _add_handler is the > correct place. If we also want to filter/check the content of the > file, then we must find a good place in Node.pm. I would also suggest > to start with the _add_handler and its derivates like share without a > source, restore, recover and so on. What sort of content inspection are you thinking of? We are already setting the mime-type to application/octet-stream if VSS considers the file "binary". > > >> >> Thanks, it's a good start. :) I can probably knock this out when I >> get some free time. > > The main problem for me is, that it takes me hours to write a simple > line in PERL. Opposed to what is written in all its documentation, > perl is not an easy language. It might be powerful, but not easy to > read and to code. > Perhaps I will play a little further. I just need to have an idea, how > to evaluate the auto-props-pattern from the hash against the filename. I won't disagree with that. Even when you understand the syntax it still takes a lot of use before your start "thinking in Perl". I chose it for its cross-platform ubiquity and string manipulation capabilities, but as I've said I would probably go with C# if I had it to do over again, now that Mono is very well established on Linux. toby From shiva at sewingwitch.com Tue Jan 23 16:07:03 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Tue Jan 23 17:07:14 2007 Subject: Perl (was: auto-properties) In-Reply-To: <45B6297B.4090304@etjohnson.us> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> <45B6297B.4090304@etjohnson.us> Message-ID: <9BD64C2E2E598DF1F7F5C88B@[10.0.0.14]> --On Tuesday, January 23, 2007 10:27 AM -0500 Toby Johnson wrote: >> The main problem for me is, that it takes me hours to write a simple >> line in PERL. Opposed to what is written in all its documentation, >> perl is not an easy language. It might be powerful, but not easy to >> read and to code. >> Perhaps I will play a little further. I just need to have an idea, how >> to evaluate the auto-props-pattern from the hash against the filename. > > I won't disagree with that. Even when you understand the syntax it still > takes a lot of use before your start "thinking in Perl". I chose it for > its cross-platform ubiquity and string manipulation capabilities, but as > I've said I would probably go with C# if I had it to do over again, now > that Mono is very well established on Linux. I use a sendmail "milter" (mail filter) written in Perl (http://mimedefang.org/) and the author indicated once that he would have preferred to write it in Tcl as it's a more robust language and Perl has some dark corners that cause problems in long-running processes. I find Perl's philosophy of more than one way to do things to itself be problematic, because I don't find a lot of guidance in why one should choose one approach over another. I do most of my coding in C++, and look longingly at Java, especially now that Sun has open-sourced it. (There are a few things it can't open-source because of 3rd party license encumbrances. But people are looking for alternatives to those parts.) Given all the tools and libraries available for Java, why would one choose C#? (I'm not religious about languages. My big fear is vendor lock-in. Mono's biggest threat seemed to be the potential for MS patent "mines".) From eli.carter at commprove.com Tue Jan 23 16:17:37 2007 From: eli.carter at commprove.com (Eli Carter) Date: Tue Jan 23 17:17:54 2007 Subject: auto-properties In-Reply-To: <45B6297B.4090304@etjohnson.us> References: <45B56B8B.8020601@nogga.de> <45B5D1AC.6050702@nogga.de> <45B6297B.4090304@etjohnson.us> Message-ID: <200701231617.37989.eli.carter@commprove.com> On Tuesday 23 January 2007 9:27 am, Toby Johnson wrote: > I won't disagree with that. Even when you understand the syntax it still > takes a lot of use before your start "thinking in Perl". I chose it for > its cross-platform ubiquity and string manipulation capabilities, but as > I've said I would probably go with C# if I had it to do over again, now > that Mono is very well established on Linux. Python! (cvs2svn as precedent) :) Eli From shiva at sewingwitch.com Tue Jan 23 16:33:46 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Tue Jan 23 17:33:54 2007 Subject: Programming languages (was: auto-properties) In-Reply-To: <200701231617.37989.eli.carter@commprove.com> References: <45B56B8B.8020601@nogga.de> <45B5D1AC.6050702@nogga.de> <45B6297B.4090304@etjohnson.us> <200701231617.37989.eli.carter@commprove.com> Message-ID: <9C278CF4CB7BA251320387A0@[10.0.0.14]> --On Tuesday, January 23, 2007 4:17 PM -0600 Eli Carter wrote: > Python! > > (cvs2svn as precedent) > > :) I often see Ruby mentioned as well, but I have no knowledge of it. My one taste of Python was less than satisfying. It appeared as an extension language in the Battlefield 2 server (for the game) and was kind of crippled out of the box. Not the language's fault, though. The idea was that one could write game administration functions in Python. From toby at etjohnson.us Tue Jan 23 17:07:05 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jan 23 18:07:11 2007 Subject: Perl In-Reply-To: <9BD64C2E2E598DF1F7F5C88B@[10.0.0.14]> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> <45B6297B.4090304@etjohnson.us> <9BD64C2E2E598DF1F7F5C88B@[10.0.0.14]> Message-ID: <45B69519.2070601@etjohnson.us> Kenneth Porter wrote: > I use a sendmail "milter" (mail filter) written in Perl > (http://mimedefang.org/) and the author indicated once that he would > have preferred to write it in Tcl as it's a more robust language and > Perl has some dark corners that cause problems in long-running processes. > > I find Perl's philosophy of more than one way to do things to itself > be problematic, because I don't find a lot of guidance in why one > should choose one approach over another. > > I do most of my coding in C++, and look longingly at Java, especially > now that Sun has open-sourced it. (There are a few things it can't > open-source because of 3rd party license encumbrances. But people are > looking for alternatives to those parts.) Given all the tools and > libraries available for Java, why would one choose C#? (I'm not > religious about languages. My big fear is vendor lock-in. Mono's > biggest threat seemed to be the potential for MS patent "mines".) Of course IANAL but the fact that C# is an ECMA standard makes me feel pretty good about coding in it. It's ironic that most Linux distros these days have better out-of-the-box support for C# than they do for Java! Of course now that Java is FOSS I'm sure that will change very quickly. I used to be firmly in the distrust-everything-Microsoft-touches camp but I've been pulling back from there over the past several years. I think they really do realize that they need to continue expanding the ecosystem around their technologies, which (as much as they wish weren't true) includes *nix interoperability. So I really don't personally think there will be any MS patent mines coming down the road for C#, any more than there is that same potential for pretty much every piece of software written by anyone these days. It's Mutually Assured Destruction all over again, with all sides building up their nuclear arsenals of patents, not wanting to push the button, but being ready if someone else does. :) So to answer your question, why I would choose C# over Java (since really they are virtually identical) is that Visual Studio 2005 is simply the best development environment I have ever worked with. I just have lots of fun coding and debugging in it, using Edit and Continue, Immediate window, the various variable visualizers, etc. The only Java-capable IDE I've used is Eclipse, which is also a great program (and has some features I wish VS did, such as the ability to override config settings on a per-project basis), but it's just not quite as polished IMO. That, and I've used C# much more simply because that's what my current employer uses. Again, this is all my personal experience; I don't want to disparage Java at all. I used to think Perl was the greatest thing since sliced bread but after working for quite a while in a strongly-typed, true OO language I see its shortcomings much better. I still love it for quick and dirty hack jobs but I doubt I'd use it again for anything as complex as this project. toby From toby at etjohnson.us Tue Jan 23 17:09:47 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jan 23 18:09:53 2007 Subject: auto-properties In-Reply-To: <200701231617.37989.eli.carter@commprove.com> References: <45B56B8B.8020601@nogga.de> <45B5D1AC.6050702@nogga.de> <45B6297B.4090304@etjohnson.us> <200701231617.37989.eli.carter@commprove.com> Message-ID: <45B695BB.7000008@etjohnson.us> Eli Carter wrote: > On Tuesday 23 January 2007 9:27 am, Toby Johnson wrote: > >> I won't disagree with that. Even when you understand the syntax it still >> takes a lot of use before your start "thinking in Perl". I chose it for >> its cross-platform ubiquity and string manipulation capabilities, but as >> I've said I would probably go with C# if I had it to do over again, now >> that Mono is very well established on Linux. >> > > Python! > > (cvs2svn as precedent) > > :) > Anyone else is welcome to rewrite this app in whatever project they wish. You can even have your own branch. :) So many languages, so little time... I've simply not had an opportunity to use Python much, aside from installing Trac and trying to figure out why mod_python kept complaining about its eggs or something. :) Or Ruby for that matter. toby From shiva at sewingwitch.com Tue Jan 23 17:37:32 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Tue Jan 23 18:37:42 2007 Subject: Programming languages In-Reply-To: <45B69519.2070601@etjohnson.us> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> <45B6297B.4090304@etjohnson.us> <9BD64C2E2E598DF1F7F5C88B@[10.0.0.14]> <45B69519.2070601@etjohnson.us> Message-ID: <57F65839542C48047A8839A5@[10.0.0.14]> --On Tuesday, January 23, 2007 6:07 PM -0500 Toby Johnson wrote: > So to answer your question, why I would choose C# over Java (since really > they are virtually identical) is that Visual Studio 2005 is simply the > best development environment I have ever worked with. I have VS2005 but haven't had a chance to do a lot with it. My client is still stuck in VC6 and my vendors haven't released VS2005 DLL's so for the moment I'm stuck in VC6 as well. (If only there was a C++ ABI so VS2005 and VC6 DLL's were interoperable....) The one danger I've found in MS IDE's (at least with VS6) is that they encourage one to use idioms that encourage lock-in, like wizards that drive one into MFC and all the clutter it involves. Right now I'm forcing myself to separate GUI and business logic. I'm using wxWidgets for GUI, which has some nice layout stuff similar to Java's, and is cross-platform, so I can rebuild for Linux soon. There's a nice RAD tool for dialogs called wxFormBuilder. I haven't played with it yet, but I've also found a Makefile generator called Bakefile that can also generate MS project files from an XML project description. I need to figure out how to reverse-engineer a Bakefile from my existing VS6 project to port the project to Linux. From vss2svn at nogga.de Tue Jan 23 18:19:25 2007 From: vss2svn at nogga.de (Dirk) Date: Tue Jan 23 19:19:33 2007 Subject: auto-properties In-Reply-To: <45B5D1AC.6050702@nogga.de> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> Message-ID: <45B6A60D.5000908@nogga.de> Hi, I don't wanted to start a language discussion here ;-), I just wanted to know, how I can evaluate a regular expression from the ini-file on a file name ;-) That's all. Concearning the languages, I think there are lots of pros and cons for almost every language. And there is always one killer feature that will decide whether you use the one or another. In the most cases it will be the language the main developers know. I will always take you some time to get used to a new language, and if you don't do work on it in a daily business it will always be a steep learning curve. I expect everybody of us learned to program BASIC, PASCAL or MODULA. Some of the higher technology languages of the past. Is anybody of us capable of writing a very simple program in one of these languages out of the box? If you don't use it, you will forget about it very fast. Dirk From toby at etjohnson.us Tue Jan 23 19:40:19 2007 From: toby at etjohnson.us (Toby Johnson) Date: Tue Jan 23 20:40:26 2007 Subject: auto-properties In-Reply-To: <45B6A60D.5000908@nogga.de> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> <45B6A60D.5000908@nogga.de> Message-ID: <45B6B903.5020301@etjohnson.us> Dirk wrote: > Hi, > > I don't wanted to start a language discussion here ;-), I just wanted > to know, how I can evaluate a regular expression from the ini-file on > a file name ;-) That's all. As far as that goes, I think having a list of all the auto-props would work better; I'm not sure how to make that work using a hash. Each expression in the ini file would need to be converted into a regular expression. It looks like Text::Glob::glob_to_regex() is exactly what would be needed[1]. (That would add another dependency; Text::Glob is not "standard" but is from CPAN so that's OK... or we could simply pull in the source for that function[2]). So then we'd need an array of hashes to contain the regex, the property, and the value: @autoprops = ( { regex => glob_to_regex('*.cpp'), prop => 'svn:eol-style', value => 'native', } ... ); (Of course this is actually loaded from the ini file). So, when you have a filename $file (where the path info has already been stripped off), you would need to test every auto-prop to see if it matches: foreach my $autoprop (@autoprops) { if ($file =~ m/$autoprop->{'regex'}/) { $node->add_prop($autoprop->{'prop'}, $autoprop->{'value'}); } } [1]http://search.cpan.org/~rclamp/Text-Glob-0.07/lib/Text/Glob.pm [2]http://search.cpan.org/src/RCLAMP/Text-Glob-0.07/lib/Text/Glob.pm From shiva at sewingwitch.com Tue Jan 23 19:57:32 2007 From: shiva at sewingwitch.com (Kenneth Porter) Date: Tue Jan 23 20:57:39 2007 Subject: auto-properties In-Reply-To: <45B6A60D.5000908@nogga.de> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> <45B6A60D.5000908@nogga.de> Message-ID: <09279C8FF46987D6E9BF0A6F@[10.0.0.14]> --On Wednesday, January 24, 2007 1:19 AM +0100 Dirk wrote: > In the most cases it will be the language the main developers know. As the "main developer" at my company ;) it's my job to research new tools. > I will always take you some time to get used to a new language, and if > you don't do work on it in a daily business it will always be a steep > learning curve. True. I know a lot of programmers who see programming as a means to some other end, and learn just enough to achieve that goal. I instead like the discipline itself, so I'm always looking into new things associated with computers and programming, which includes languages, libraries, and techniques. One reason I want to avoid vendor lock-in is that it prevents me from taking advantage of new technologies if they're offered by a competitor. From stephen.lee at hexagonmetrology.com Wed Jan 24 06:22:01 2007 From: stephen.lee at hexagonmetrology.com (Stephen Lee) Date: Wed Jan 24 07:22:15 2007 Subject: A horrible and hacky filter for the "false orphan" problem... Message-ID: <45B74F69.7040604@wilcoxassoc.com> As I mentioned before, the 0.11.0 alpha1 (which still seems to be the latest .exe version actually available - I never got as far as sorting out the required perl environment under windows) got somewhat confused by my database... Some files, and even entire projects were in the orphaned folder when they should have been in other locations, mainly it seemed due to confusion caused by multiple files/projects that had used the same name at different points in time, or had been renamed back and forth etc. Others were "genuine orphans" - e.g. files that had been branched, and one of the branches deleted, or had been initially created in a project that was later destroyed after sharing the files to another project. While I could have fixed it up afterwards in Subversion, this would have needed repeating for every labelled version (in my case entire subprojects that had been created very early in the development were incorrectly orphaned, and were thus missing from every labelled version) The attached "filterorphan.cpp" is run as a filter on the dumpfile. I used the equivalent of the following command sequence for each iteration: cl filterorphan.cpp filterorphan.exe < vss2svn-dumpfile.txt > dumpfile-filtered.txt rmdir /s /q importtest svnadmin create importtest svnadmin load importtest < dumpfile-filtered.txt I've trimmed most of my database-specific content from the .cpp file, but left a few in to give an idea of how it would work. I had approx 300 orphans to deal with of which just over a dozen were "false orphans", and most of the others were easily identified in large batches due to being created/branched in a subsequently destroyed project. The code is not necessarily the nicest you'll ever have seen (there's some stuff that it ends up doing 2 or 3 different ways), but was "good enough" and "does the job" for me... it is provided on an as-is basis for anyone who needs something similar and wants to tidy it up, or for anyone who wants a hint about the kind of stuff that the conversion script got wrong... First time it is run, it will output to stderr a list of all orphans in a format than can be copy-pasted into the SetupReplacements function, and filled in as needed. I found it easiest to classify each "true orphan" into a subfolder of "orphaned" rather than the one-per-file orphaned subfolder that the script produces, and put the "false orphans" in the correct place. I was also working with the datacache.PhysicalAction.tmp.txt and datacache.VssAction.tmp.txt loaded into excel (and the former sorted by the timestamp column), which helps a lot with figuring out which file was which, and sorting out the true orphans from the false. On later runs, the output to stderr will also include indications of progress and actions taken depending on which lines are uncommented at the time... In basic terms the file works by reading the input file one byte at a time, and matching it against a number of potential triggers. If your database is much bigger than mine (which was a dumpfile <1GB) you might want something a bit more streamlined, or to translate the ideas into a language that's more efficient for stream manipulation (I used C-style C++ solely because the IDE and programmer's familiarity was already there. Given a freer choice it would have been a relatively poor choice for this problem). Other tools I might otherwise have used simply couldn't cope with a file that size, or with the text/binary mix. "orphaned/_" was originally the only trigger, and the others were added incrementally as the errors in the svnadmin load command or other discrepancies dictated... for example suppressing creation of a dir that had already been created, creating a dir that did did not exist yet, duplicating an action on one shared copy of a file on another that the script had "forgotten" about, etc... If anyone actually wants any other part of this program explaining feel free to contact me, but it is essentially the "final" (for me) iteration of a make-it-up-as-you-go-along cycle, at the end of which most of the head and labelled revisions were correct. Comments are unfortunately fairly sparse, being targeted mainly at "me trying to figure out what I was doing 1 or 2 months later". -- Stephen Lee Software Engineer, Vision Group - Pro-Measure Leader Wilcox Associates Inc. (U.K.) -------------- next part -------------- #include #include #include #include #include struct ReplaceNode { inline operator const char *(void) const { return Text; } const char *Text; bool CreatedDir; bool CreatedParent; bool Used; }; enum TriggerType { Type_MoveOrphan, Type_SuppressCreate, Type_OneShotReplace, Type_NoMove, Type_OneShotDupe, Type_MultiReplace, Type_MultiDupe, }; struct TriggerNode { const char *TriggerText; const char *CopyText; union { ReplaceNode *Node; int SkipCount; }; TriggerType Type; }; // increase max number of triggers as needed, // but NB current implementation cannot cope with DIFFERENT text starting with same letter. // (same text with different Node is ok though) TriggerNode Triggers[200]; int TriggerCount=0; int RevSubst[100000]; int MaxRevSeen=0; char DupeBuffer[1000000]; int DupeIndex=0; bool DoneDupePrint; ReplaceNode ReplaceIndex[26][26][26][26]; struct { const char *ReplaceText; const char *NextText; const char *NextReplace; // of just the NextText for a non-dupe, or of entire path for dupe. bool Dupe; char SkipCount; } SpecialReplace[] = { { "Plugins","/Gsc200s/Gsc200s.rc2","/Gsc200s/res/Gsc200s.rc2",false,0 }, // snip other SpecialReplace entries. }; void Replace(const char *FileID,const char *ReplaceText,bool DirAlreadyExists=true,const char *ReplaceText2=NULL,const char *ReplaceTrigger=NULL,const char *ParentCreate=NULL) { ReplaceNode *Node=&ReplaceIndex[FileID[0]-'A'][FileID[1]-'A'][FileID[2]-'A'][FileID[3]-'A']; Node->Text=ReplaceText; Node->CreatedDir=DirAlreadyExists; Node->CreatedParent=ParentCreate==NULL; if(ReplaceText2) { Triggers[TriggerCount].Node=Node; Triggers[TriggerCount].CopyText=ReplaceText2; Triggers[TriggerCount].TriggerText=ReplaceTrigger?ReplaceTrigger:ReplaceText2; Triggers[TriggerCount].Type=Type_MoveOrphan; TriggerCount++; } if(!DirAlreadyExists && strncmp(ReplaceText,"orphaned",8)!=0) { if(ParentCreate) { if(strncmp(ParentCreate,ReplaceText,strlen(ParentCreate))!=0) fprintf(stderr,"!!!!!! BAD PARAMETER !!!!!!\n"); // suppress parent when it gets created Triggers[TriggerCount].Node=NULL; Triggers[TriggerCount].CopyText=NULL; Triggers[TriggerCount].TriggerText=ParentCreate; Triggers[TriggerCount].Type=Type_SuppressCreate; TriggerCount++; } // suppress when it does get created. Triggers[TriggerCount].Node=NULL; Triggers[TriggerCount].CopyText=NULL; Triggers[TriggerCount].TriggerText=ReplaceText; Triggers[TriggerCount].Type=Type_SuppressCreate; TriggerCount++; } } void OneShotReplace(const char *Trigger,const char *ReplaceText,int SkipCount=0) { Triggers[TriggerCount].SkipCount=SkipCount; Triggers[TriggerCount].CopyText=ReplaceText; Triggers[TriggerCount].TriggerText=Trigger; Triggers[TriggerCount].Type=Type_OneShotReplace; TriggerCount++; } void OneShotDupe(const char *Trigger,const char *ReplaceText,int SkipCount=0) { Triggers[TriggerCount].SkipCount=SkipCount; Triggers[TriggerCount].CopyText=ReplaceText; Triggers[TriggerCount].TriggerText=Trigger; Triggers[TriggerCount].Type=Type_OneShotDupe; TriggerCount++; } void MultiReplace(const char *Trigger,const char *ReplaceText,int SkipCount=0) { Triggers[TriggerCount].SkipCount=SkipCount; Triggers[TriggerCount].CopyText=ReplaceText; Triggers[TriggerCount].TriggerText=Trigger; Triggers[TriggerCount].Type=Type_MultiReplace; TriggerCount++; } void MultiDupe(const char *Trigger,const char *ReplaceText,int SkipCount=0) { Triggers[TriggerCount].SkipCount=SkipCount; Triggers[TriggerCount].CopyText=ReplaceText; Triggers[TriggerCount].TriggerText=Trigger; Triggers[TriggerCount].Type=Type_MultiDupe; TriggerCount++; } void NoMove(const char *Source,const char *Dest) { Triggers[TriggerCount].Node=NULL; Triggers[TriggerCount].CopyText=Source; Triggers[TriggerCount].TriggerText=Dest; Triggers[TriggerCount].Type=Type_NoMove; TriggerCount++; } void SetupReplacements(void) { // first parameter to Replace MUST begin with 4 capital letters. Replace("AAAAAAAA","MyProject"); // Replace("CAAAAAAA","orphaned/DongleCheck",false); // /DongleCheck.cpp Replace("DAAAAAAA","orphaned/DongleCheck"); // /DongleCheck.dsp Replace("EAAAAAAA","orphaned/DongleCheck"); // /DongleCheck.h Replace("FAAAAAAA","orphaned/DongleCheck"); // /DongleCheck.rc Replace("GAAAAAAA","orphaned/DongleCheck"); // /DongleCheckDlg.cpp Replace("HAAAAAAA","orphaned/DongleCheck"); // /DongleCheckDlg.h Replace("IAAAAAAA","orphaned/DongleCheck"); // /ReadMe.txt Replace("JAAAAAAA","orphaned/DongleCheck"); // /resource.h Replace("KAAAAAAA","orphaned/DongleCheck"); // /StdAfx.cpp Replace("LAAAAAAA","orphaned/DongleCheck"); // /StdAfx.h Replace("EKAAAAAA","orphaned/DongleCheck/res",false); // /cross.bmp Replace("FKAAAAAA","orphaned/DongleCheck/res"); // /DongleCheck.ico Replace("GKAAAAAA","orphaned/DongleCheck/res"); // /DongleCheck.rc2 Replace("HKAAAAAA","orphaned/DongleCheck/res"); // /tick.bmp // snip various other Replace(...) lines... Replace("CJBAAAAA","Plugin/Gsc200s",true,"Plugins/Gsc200s","Plugins"); // /Gsc200sPlugin.cpp // snip many more Replace(...) lines... // special triggers NoMove("Plugins/Flyte","Flyte"); RevSubst[302]=370; OneShotReplace("InstallImage/Install/v2.26.ipr","InstallImage/Install/V2.26.ipr"); OneShotReplace("InstallImage/Install/v2.26.ipr","InstallImage/Install/V2.26.ipr"); // snip various fixups of stuff that failed to get put in the Plugins/Flyte project // mainly OneShotReplace and OneShotDupe lines, and some Multi... lines. } int main(void) { _setmode( _fileno( stdin ), _O_BINARY ); _setmode( _fileno( stdout ), _O_BINARY ); memset(ReplaceIndex,0,sizeof(ReplaceIndex)); memset(RevSubst,0,sizeof(RevSubst)); SetupReplacements(); const char *DirCreateText="\n" "Node-kind: dir\n" "Node-action: add\n" "Prop-content-length: 10\n" "Content-length: 10\n" "\n" "PROPS-END\n" "\n" "\n" "Node-path: "; int DirCreateTextLen=strlen(DirCreateText); const char NodePathTrigger[]="Node-path:"; const char NodeRevTrigger[]="Node-copyfrom-rev: "; const char NextRevTrigger[]="Revision-number:"; bool NodePathFlag; bool LastSpace=false; const char Prefix[]="orphaned/_"; const char *TriggerStart=NULL; const char *TriggerPtr=NULL; const char *Marker="!!!!!!!this should not appear in the output file!!!!!!!"; char c; // int dbgcount=50; while(fread(&c,1,1,stdin)) // horribly slow but simple 1 char at a time... { // if(dbgcount-->0) // fprintf(stderr,"Got char: %d ('%c')\n",c,c); if(TriggerPtr && c==*TriggerPtr) { TriggerPtr++; if(!*TriggerPtr) { if(DupeIndex && TriggerStart!=NodePathTrigger && TriggerStart!=NextRevTrigger && TriggerStart!=NodeRevTrigger) { fprintf(stderr,"Matched trigger in Dupe mode: %s\n",TriggerStart); return 1; } // complete trigger matched. if(TriggerStart==Prefix) { // matched prefix. next 8 chars are the file ID, followed by the original filename. char FileID[9]; fread(&FileID,1,8,stdin); FileID[8]='\0'; int ValidChars; for(ValidChars=0;ValidChars<8;ValidChars++) { // fprintf(stderr,"FileID[%d]: %d ('%c')\n",ValidChars,FileID[ValidChars],FileID[ValidChars]); if(FileID[ValidChars]<'A' || FileID[ValidChars]>'Z') break; /* { FileID[8]='\0'; fprintf(stderr,"Sorry I can't cope with %s%s\n",Prefix,FileID); return 1; // TODO replace the above with code to ignore the false trigger if necessary. }*/ } ReplaceNode *Node=ValidChars<8?NULL:&ReplaceIndex[FileID[0]-'A'][FileID[1]-'A'][FileID[2]-'A'][FileID[3]-'A']; const char *ReplaceText=Node?Node->Text:NULL; if(ReplaceText) { if(ReplaceText==Marker) { fwrite(&Prefix,1,TriggerPtr-Prefix,stdout); fwrite(FileID,1,8,stdout); } else { if(!Node->CreatedParent) { // need to create the parent of ReplaceText first. int LastSep=0; for(int i=0;ReplaceText[i];i++) if(ReplaceText[i]=='/') LastSep=i; fwrite(ReplaceText,1,LastSep,stdout); fwrite(DirCreateText,1,DirCreateTextLen,stdout); Node->CreatedParent=true; // avoid re-creating when it is updated, copied, etc. } fwrite(ReplaceText,1,strlen(ReplaceText),stdout); Node->Used=true; // now check if this was a dir creation, and if so if we already created the same // replacement dir already. If so skip ahead as next section will always be the // node creation within the dir. fread(&c,1,1,stdin); // fprintf(stderr,"Next char: %d ('%c')\n",c,c); if(c=='\n') { // this is referring just to the orphaned dir itself. next characters should match // the following text: int DirCreateMatchLen=1; while(DirCreateMatchLen250) { LabelPathText[LabelPathTextLen]='\0'; fprintf(stderr,"PathTriggerText not found after %c chars. Looking for %s in:\n%s", LabelPathTextLen+PathTriggerTextLen,PathTriggerText,LabelPathText); fwrite(DirCreateText,1,DirCreateTextLen,stdout); fwrite(LabelPathText,1,LabelPathTextLen,stdout); fwrite(PathTriggerText,1,PathTriggerMatchLen,stdout); fwrite(&c,1,1,stdout); break; } else { if(PathTriggerMatchLen) { memcpy(LabelPathText+LabelPathTextLen,PathTriggerText,PathTriggerMatchLen); LabelPathTextLen+=PathTriggerMatchLen; PathTriggerMatchLen=0; } LabelPathText[LabelPathTextLen++]=c; } } if(!(DirCreateMatchLenCreatedDir) { // write out DirCreateText, but replacing the last section with ReplaceText... fwrite(DirCreateText,1,DirCreateTextLen,stdout); fwrite(LabelPathText,1,LabelPathTextLen,stdout); fwrite(ReplaceText,1,strlen(ReplaceText),stdout); // Node->CreatedDir=true; // fprintf(stderr,"DirCreateText Matched and output! Label:'%s'\n",LabelPathText); } // else // fprintf(stderr,"DirCreateText Matched and suppressed label:'%s'\n",LabelPathText); } } } else { int Index; char Buffer[100]; int Count=0; Buffer[Count++]=c; for(Index=0;Index %s ",SpecialReplace[Index].NextText,SpecialReplace[Index].NextReplace); if(SpecialReplace[Index].Dupe) { if(NodePathFlag) { DupeBuffer[0]=' '; DupeIndex=1+strlen(strcpy(&DupeBuffer[1],SpecialReplace[Index].NextReplace)); // fprintf(stderr,"[DupeStart]\n"); } // else // fprintf(stderr,"[Ignored]\n"); } else { fwrite(SpecialReplace[Index].NextReplace,1,strlen(SpecialReplace[Index].NextReplace),stdout); Count=0; } break; } } fread(&Buffer[Count++],1,1,stdin); if(Buffer[Count-1]!=SpecialReplace[Index].NextText[Count-1]) break; } } } } if(Count) fwrite(Buffer,1,Count,stdout); // else // fprintf(stderr,"...Done SpecialReplace!\n"); } } } else { fwrite(&Prefix,1,TriggerPtr-Prefix,stdout); fwrite(FileID,1,8,stdout); if(ValidChars==8) { char LeafName[256]; int LeafLen=0; while(LeafLen1) { fprintf(stderr," Replace(\"%s\",\"????\"); // %s",FileID,LeafName); Replace(FileID,Marker); } } } } else if(TriggerStart==NodePathTrigger || TriggerStart==NextRevTrigger) { if(DupeIndex) { fwrite(NodePathTrigger,1,strlen(NodePathTrigger),stdout); fwrite(DupeBuffer,1,DupeIndex,stdout); // fprintf(stderr,"Done Dupe!\n"); DupeIndex=0; } fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); NodePathFlag=(TriggerStart==NodePathTrigger); } else if(TriggerStart==NodeRevTrigger) { if(DupeIndex) { memcpy(&DupeBuffer[DupeIndex],TriggerStart,TriggerPtr-TriggerStart); DupeIndex+=TriggerPtr-TriggerStart; } fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); char Buffer[8]; int i; for(i=0;i<7;i++) { fread(&Buffer[i],1,1,stdin); if(Buffer[i]=='\n') { i++; break; } else if(Buffer[i]<'0' || Buffer[i]>'9') { Buffer[i+1]='\0'; fprintf(stderr,"Non-numeric revision! (starting \"%s\")\n",Buffer); return 1; } } int Rev; if(sscanf(Buffer,"%d\n",&Rev)) { if(Rev>MaxRevSeen) MaxRevSeen=Rev; else if(RevSubst[Rev] && RevSubst[Rev] %s",Rev,Buffer); } } else fprintf(stderr,"sscanf failed!!!\n"); fwrite(Buffer,1,i,stdout); if(DupeIndex) { memcpy(&DupeBuffer[DupeIndex],Buffer,i); DupeIndex+=i; } } else { int i=0; bool ActionTaken=false; while(iText=Triggers[i].CopyText; break; case Type_SuppressCreate: if(!ActionTaken) { if(Triggers[i].SkipCount) { fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); Triggers[i].SkipCount--; RemoveTrigger=false; } else { fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); fwrite("-suppressed",1,11,stdout); } } else { fprintf(stderr,"ActionTaken before a SuppressCreate\nTriggered on '%s', type %d\n",Triggers[i].TriggerText,Triggers[i].Type); return 1; } break; case Type_MultiReplace: RemoveTrigger=false; // follow through case Type_OneShotReplace: if(!ActionTaken) { if(Triggers[i].SkipCount) { fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); Triggers[i].SkipCount--; RemoveTrigger=false; } else fwrite(Triggers[i].CopyText,1,strlen(Triggers[i].CopyText),stdout); ActionTaken=true; } else RemoveTrigger=false; // could be a second "OneShotReplace" stacked up to make a TwoShotReplace, etc... break; case Type_MultiDupe: RemoveTrigger=false; // follow through case Type_OneShotDupe: if(!ActionTaken) { if(Triggers[i].SkipCount) { Triggers[i].SkipCount--; RemoveTrigger=false; } else { fprintf(stderr,"OneShotDupe: %s -> %s\n",Triggers[i].TriggerText,Triggers[i].CopyText); DupeBuffer[0]=' '; DupeIndex=1+strlen(strcpy(&DupeBuffer[1],Triggers[i].CopyText)); } fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); ActionTaken=true; } else RemoveTrigger=false; // could be a second "OneShotDupe" stacked up to make a TwoShotDupe, etc... break; case Type_NoMove: // prevent a 'move' from occuring... do this by creating a OneShotReplace so that the // delete action applies to the intended target dir rather than the source dir. // this means that the copy still occurs, as does the delete, but delete has a different target. if(!ActionTaken) { fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); ActionTaken=true; } Triggers[i].Node=NULL; { register const char *Temp=Triggers[i].CopyText; Triggers[i].CopyText=Triggers[i].TriggerText; Triggers[i].TriggerText=Temp; } Triggers[i].Type=Type_OneShotReplace; // TriggerCount++; RemoveTrigger=false; break; default: fprintf(stderr,"SOMEONE FORGOT TO WRITE THIS BIT!!!!\nTriggered on '%s', type %d\n",Triggers[i].TriggerText,Triggers[i].Type); return 1; } if(!RemoveTrigger) i++; else if(i<--TriggerCount) memmove(&Triggers[i],&Triggers[i+1],sizeof(*Triggers)*(TriggerCount-i)); } else i++; } } TriggerPtr=TriggerStart=NULL; } } else { if(TriggerPtr!=TriggerStart) { int CurrentOffset=TriggerPtr-TriggerStart; // look for an alternative trigger starting with the same letter(s) if(strncmp(TriggerStart,NodeRevTrigger,CurrentOffset)==0 && c==NodeRevTrigger[CurrentOffset]) { TriggerStart=NodeRevTrigger; TriggerPtr=TriggerStart+CurrentOffset; } else if(NodePathFlag) { for(int i=0;iUsed)) { TriggerStart=Triggers[i].TriggerText; TriggerPtr=TriggerStart+CurrentOffset; break; } } } if(c!=*TriggerPtr) { fwrite(TriggerStart,1,TriggerPtr-TriggerStart,stdout); if(DupeIndex) { memcpy(&DupeBuffer[DupeIndex],TriggerStart,TriggerPtr-TriggerStart); DupeIndex+=TriggerPtr-TriggerStart; } TriggerPtr=TriggerStart=NULL; NodePathFlag=false; } } if(!TriggerStart) { if(LastSpace && c==Prefix[0]) TriggerPtr=TriggerStart=Prefix; else if(c==NodePathTrigger[0]) TriggerPtr=TriggerStart=NodePathTrigger; else if(c==NextRevTrigger[0]) TriggerPtr=TriggerStart=NextRevTrigger; else if(NodePathFlag) { for(int i=0;iUsed)) { TriggerPtr=TriggerStart=Triggers[i].TriggerText; break; } } } if(TriggerPtr && c==*TriggerPtr) TriggerPtr++; else { fwrite(&c,1,1,stdout); if(DupeIndex) DupeBuffer[DupeIndex++]=c; LastSpace=c==' '; if(!LastSpace) NodePathFlag=false; } } } while(TriggerCount-->0) { fprintf(stderr,"Unused Trigger: \"%s\" (Type %d, Copy '%s' to %p)\n", Triggers[TriggerCount].TriggerText,Triggers[TriggerCount].Type,Triggers[TriggerCount].CopyText,Triggers[TriggerCount].Node); } return 0; } From vss2svn at nogga.de Wed Jan 24 08:54:02 2007 From: vss2svn at nogga.de (Dirk) Date: Wed Jan 24 09:53:01 2007 Subject: auto-properties In-Reply-To: <45B6B903.5020301@etjohnson.us> References: <45B56B8B.8020601@nogga.de> <45B56E2E.2000800@etjohnson.us> <45B5D1AC.6050702@nogga.de> <45B6A60D.5000908@nogga.de> <45B6B903.5020301@etjohnson.us> Message-ID: <45B7730A.30502@nogga.de> Hi Toby > > As far as that goes, I think having a list of all the auto-props would > work better; I'm not sure how to make that work using a hash. Each > expression in the ini file would need to be converted into a regular > expression. It looks like Text::Glob::glob_to_regex() is exactly what > would be needed[1]. (That would add another dependency; Text::Glob is > not "standard" but is from CPAN so that's OK... or we could simply > pull in the source for that function[2]). Thanks, this was exactly what I was looking for. So perhaps ... , if I find some more time tonight .... ;-) Dirk From toby at etjohnson.us Wed Jan 24 09:20:12 2007 From: toby at etjohnson.us (Toby Johnson) Date: Wed Jan 24 10:20:18 2007 Subject: A horrible and hacky filter for the "false orphan" problem... In-Reply-To: <45B74F69.7040604@wilcoxassoc.com> References: <45B74F69.7040604@wilcoxassoc.com> Message-ID: <45B7792C.807@etjohnson.us> Stephen Lee wrote: > As I mentioned before, the 0.11.0 alpha1 (which still seems to be the > latest .exe version actually available - I never got as far as sorting > out the required perl environment under windows) got somewhat confused > by my database... > > Some files, and even entire projects were in the orphaned folder when > they should have been in other locations, mainly it seemed due to > confusion caused by multiple files/projects that had used the same > name at different points in time, or had been renamed back and forth etc. > > Others were "genuine orphans" - e.g. files that had been branched, and > one of the branches deleted, or had been initially created in a > project that was later destroyed after sharing the files to another > project. > > While I could have fixed it up afterwards in Subversion, this would > have needed repeating for every labelled version (in my case entire > subprojects that had been created very early in the development were > incorrectly orphaned, and were thus missing from every labelled version) > > > The attached "filterorphan.cpp" is run as a filter on the dumpfile. I > used the equivalent of the following command sequence for each iteration: Thanks Stephen, I have created a "contrib" directory and checked your code in there. I added your notes into a README.txt file. Yes, 0.11.0-a1 is still the latest .exe version available; there has not been much activity on this project in the past several months but now that we're seeing some issues get cleared up I'll probably create another one shortly. toby From toby at etjohnson.us Wed Jan 24 09:29:51 2007 From: toby at etjohnson.us (Toby Johnson) Date: Wed Jan 24 10:29:54 2007 Subject: Ticket triage? In-Reply-To: <45B5D014.5060103@nogga.de> References: <316CAD78F09E620CDF24D2D8@[10.170.7.6]> <459D14BC.5050302@etjohnson.us> <80AFE6B50EB540B69047B003@[10.170.7.6]> <459D2E62.2030201@etjohnson.us> <84277F98D0BBEA8DCE307CC1@[10.170.7.6]> <459D3257.6080703@etjohnson.us> <45A03668.8000403@bezeqint.net> <45A15346.5090309@nogga.de> <45AEAAEF.5040607@etjohnson.us> <45B545CE.2040901@nogga.de> <45B54ED1.7060203@etjohnson.us> <45B5528D.40906@nogga.de> <45B56CAB.9020802@etjohnson.us> <45B5D014.5060103@nogga.de> Message-ID: <45B77B6F.4090402@etjohnson.us> Dirk wrote: > >> >> Thanks Dirk, glad to see you have some free time lately :) Who knows, >> maybe I'll even find some time to knock out a few tickets myself! > > I think a few tickets that are still open are already closed. We > should have a look at the list. > > Dirk Yes, I have been closing any I find that are now invalid, but please feel free to do so yourself. I'm not familiar w/ all the tickets currently open. From grace.go at syncvoice.com Thu Jan 25 14:42:21 2007 From: grace.go at syncvoice.com (Grace Go) Date: Thu Jan 25 15:42:33 2007 Subject: Error when I ran vss2svn In-Reply-To: <45806E05.1030406@nogga.de> Message-ID: <110gkpj1nbnzcgj.250120071242@syncvoice.com> Is there any way to run svnadmin load c:\repository < vss2svn -dumpfile.txt