Suggested vss2svn workflow (was RE: Invalid change ordering?)

[Scott Nowell]

Wow!  I'm going to have to hire an IT person.

Your procedure sure makes sense though.  Thanks.

Scott 

> -----Original Message-----
> From: vss2svn-users-bounces at lists.pumacode.org 
> [mailto:vss2svn-users-bounces at lists.pumacode.org] On Behalf 
> Of Jonathan Perret
> Sent: Wednesday, February 07, 2007 4:57 AM
> To: Vss2Svn Users
> Subject: Suggested vss2svn workflow (was RE: Invalid change ordering?)
> 
> > Regarding the "live" database, you can simply make a file 
> copy of the 
> > VSS files to your local computer and run the conversion 
> from there. If 
> > you do it in a silent minute, during the night, you could 
> probaply get 
> > a pristine copy.
> 
> Definitely ! A copy of the database should be the first thing 
> you do to begin your migration. Actually, in case anyone is 
> interested, here is the complete workflow I used for our migration :
> 
> 1. Choose a powerful, dedicated machine to run the migration. 
> It can be
>   your own laptop, or another server just as long as it is not the VSS
>   server. I used our automated build server since it already had Perl
>   and is a pretty beefy box that is somewhat underused. From 
> this point on
>   I'm assuming a Windows box was selected.
> 
> 2. Make sure the following software is available on that machine :
>  - SourceSafe 6.0d (please avoid VSS 8, I'm sorry to say this but they
>    actually managed to make it less stable). Make sure SS.exe and 
>    ANALYZE.exe are on the PATH.
>  - ROBOCOPY.EXE, you can get this from the Windows Resource Kits, or
>    many other places on the web. Make sure it is on the PATH too.
>  - vss2svn (preferably in Perl source form, so that you are ready for
>    a quick patch if anything goes wrong) and its dependencies.
>  - the latest Subversion, make sure it is in the PATH too.
> 
> 3. Log in to that machine using an account that has read-only 
> access to
>   the VSS share. I'll actually be altering the VSS database before the
>   migration and I'm a bit paranoid about accidentally destroying our
>   production database. Notice that this read-only access prevents the
>   SourceSafe client from accessing the VSS database, but that's
>   actually a good thing.
> 
> 4. Create a local copy of your VSS database using ROBOCOPY :
>  ROBOCOPY /MIR \\vssserver\vss C:\svnmigration\vsscopy
> 
> 5. Open srcsafe.ini in the VSS db copy, make sure to remove
>   any references to shadow folders and a journal file : again
>   we don't want local operations on this copy to disrupt the
>   production environment.
> 
> 5. Create a directory somewhere, call it something like
>   "C:\svnmigration\migration1".
> 
>   You'll be doing several migrations and you may want to compare the
>   results from one to another so it's a good idea to archive 
> the attempts.
>   I went further and copied the whole of vss2svn under this directory,
>   this allowed me to also archive what little tweaks I had made to the
>   script for each attempt.
> 
> 6. If you want to keep your hair through your migration, a key
>   word is "automate" ! Create a script using whatever systems 
> automation
>   language you are comfortable with. I guess Perl would be a logical
>   choice, but I selected CMD scripts (batch files) since I didn't know
>   enough Perl to do what I wanted. Your script should do the 
> following :
>    a. synchronize the data in the local copy of the VSS database :
> 
> 	SET SSDIR=C:\svnmigration\vsscopy
> 	ROBOCOPY /MIR \\vssserver\vss\data %SSDIR%\data
> 
>      (notice I synchronized only the data subdirectory since I want
>       to keep the local modifications to srcsafe.ini that I did above)
>      (notice also that the /MIR switch to ROBOCOPY makes it remove
>       files that are no longer present in the VSS db, and also
>       avoids copying files that have not been modified, which means
>       this operation should run pretty fast for each migration)
> 
>    b. "SS Destroy" any projects/files you don't want to migrate :
> 
> 	SS Destroy -I-Y $/ProjectIDontWant
>     
> 	(-I-Y means "shut up and do it" to SS.exe)
> 
> 	I have found that destroying projects in a copy leads to
> 	cleaner migrations than the alternative which is to
> 	backup/restore only selected projects.
> 
>       Some of the projects/files that you don't want may have 
> been deleted,
> 	but not purged. You need to recover them first :
> 
> 	SS Recover $/MyProject/HugeFileThatWasCheckedInByMistake.zip
> 	SS Destroy -I-Y
> $/MyProject/HugeFileThatWasCheckedInByMistake.zip
> 
>    c. Run ANALYZE on the database copy :
> 	
> 	rmdir /S/Q %SSDIR%\data\backup
> 	ANALYZE -f -i- -v4 %SSDIR%\data
> 	copy /Y %SSDIR%\data\backup\analyze.log .
> 
> 	(ANALYZE flags : force repair, no interaction, deep analysis)
> 
>    d. Run vss2svn :
> 	
> 	perl -Ivss2svn vss2svn\vss2svn.pl --ssphys 
> vss2svn\ssphys.exe --vssdir %SSDIR% --verbose --timing > 
> vss2svn.log 2>&1
> 
> 	(this assumes you made a local copy of vss2svn)
> 
>    e. Create and load the SVN repository :
> 
> 	rmdir /S/Q repos
> 	svnadmin create repos
> 	svnadmin load repos < vss2svn-dumpfile.txt > svnadmin.log 2>&1
> 
>    f. Do any post-migration cleanup in svn :
> 
>       SET SVNREPOS=file:///%CD:\=/%/repos
>       svn mv %SVNREPOS%/MainBranch /trunk
>       svn rm %SVNREPOS%/OldProject
> 
> 	(see point 10 for why it is sometimes better to remove 
> a directory from
> 	the SVN repository than to purge it from VSS beforehand)
> 
> 7. Launch your script, making sure to redirect all output to 
> a log file
>    that you can analyze later for errors.
> 
> 	migrate.cmd > migrate.log 2>&1
> 
> 8. Let time pass for a bit. If you want to know what the migration is
>    doing at any point, you may find 
> http://tailforwin32.sourceforge.net/
>    useful to look at the log files as they are being appended to.
> 
> 9. Once the migration is complete, take a look at the results. What
>    I did was launch an svnserve on the migration machine and browse
>    the repository from my laptop using TortoiseSVN.
> 
> 10. You'll find something you don't like about the migration results.
>    No problem, just create a "migration2" directory, copy vss2svn and
>    your script there, make any fixes you need and start again.
> 
>  Some random things to look out for :
> 
>  - It is a good idea to grep for ERROR in the vss2svn.log file, but
>    don't get frightened by every match : unfortunately 
> vss2svn does not
>    try very hard currently to rank the errors from benign to fatal.
> 
>  - Are there "too many" files in /orphaned ? You may have had 
> a heavy hand in step 6b above (destroying projects). Let's 
> say for example a file was initially created in 
> $/Version1/file.txt, and then $/Version1 was shared/branched 
> to $/Version2. If you destroy $/Version1, $/Version2/file.txt 
> is orphaned ! It is much better to keep $/Version1 in the 
> migration, then add "svn rm /Version1" to step 6f if you 
> truly want to hide that old project.
>    In short, try to only destroy truly irrelevant projects.
> 
>  - What are the biggest revisions in your new SVN repository 
> ? This is easy to find by peeking into repos\db\revs. Most of 
> these files should be pretty small, any that exceeds a few 
> megabytes is suspicious (well, depending on your data of 
> course, if you were handling large assets in VSS it could be 
> just fine). In my repository only 85 revisions out of 24500 
> exceed one megabyte.
>    To see the details of a "suspicious" revision :
> 
> 	svn log svn://path/to/repository/root/ -v -rREV
> 
>    This could help you find the instances of 
> HugeFileThatWasCheckedInByMistake.zip that bloat your 
> repository for no good reason. Go then to step 6b and add 
> such files to the "black list".
> 
> That's it ! I hope these suggestions are useful to someone 
> going through a migration.
> Perhaps the vss2svn wiki should contain information of this kind ?
> 
> Cheers,
> --Jonathan
> 
> _______________________________________________
> vss2svn-users mailing list
> Project homepage:
> http://www.pumacode.org/projects/vss2svn/
> Subscribe/Unsubscribe/Admin:
> http://lists.pumacode.org/mailman/listinfo/vss2svn-users-lists
.pumacode.org
> Mailing list web interface (with searchable archives):
> http://dir.gmane.org/gmane.comp.version-control.subversion.vss
2svn.user
> 
>