Got Git? Rails, Plugins, and Submodules Not a Go Go 5
Posted Tuesday, May 20, 2008 08:43 by chaupt
Sorry, it was a really long night, so I’m a bit Ga Ga. This is a summary of some of the challenges we’ve had with our Git migration and attempting to use submodules for vendor/rails and various plugins. We’ve run in to problems switching and merge between branches with and without submodules. Once I work this all out, I’ll write up a “real” article for BuildingWebApps.com.
For our current project, we’ve made the migration over to the Git distributed version control system. I’ve been using Git for tracking 3rd party open source projects, as well as using Mercurial. I’m still learning the ropes. Git’s support for branching was the main selling point for us. We’ve been using Subversion for a long time with great success, but the biggest weakness has been how much a pain it is to have multiple branches and deal with multi-way merges. While switching to Git requires a bit of a learning curve, it isn’t too bad.
Git makes working with multiple branches a relative breeze. Last week, I decided to give the submodule feature a try for tracking Ruby on Rails (vendor/rails) Edge, as well as a to migrate various plugins over. We used Piston in Subversion to good effect, and submodules seemed adequate as a replacement. There are numerous articles about setting up submodules, a couple of good ones are here and here.
Our scenario may or may not be that unique. We have a master branch (and other working branches) that represent our “version 1” code tree. We are doing a large refactoring in a set of branches for “version 2”. The two lines are running in parallel, and we occasionally merge/rebase from version 1 over to version 2. Version 1 is frozen around Rails 2.0.2. Version 2 is tracking Edge. I’ve set up version 2 to use submodules for vendor/rails as well as a number of our plugins. We are using Git 1.5.5.x at the moment.
The problem appears to be when you swap between branches of version 1 and version 2 (those without submodule and those with, respectively). There may be other problems, but we are tracking this down (interestingly, I have seen different results between Git 1.5.4 and 1.5.5, bugs?).
Starting with a clean clone of the remote repository (we are using GitHub quite happily), all starts out good. The master branch is clean and version 1 looks good.
I then switch to version 2 with a git checkout --track -b version2 origin/version2. Remember that version 2 is using submodules for vendor/rails and some plugins (e.g. rspec, rspec-rails, acts_as_versioned, etc.). I see delete messages for vendor/rails and the submoduled plugins. Interestingly, if I then do a git status, I see two entries (one for vendor/rails, another for one plugin—acts_as_versioned in this example). I don’t see the other plugins. The correlation here appears to be that the delete status only exists for those plugins that exist currently on the master (version 1) branch.
If I do git submodule init next, I see entries for all of my submodules being “registered”. If I then immediately do a git submodule update, I see the standard updating/downloading type messages and get the correct versions. Now if I git status, all on the version2 branch appears “clean”.
At this point, if I want to switch back to master (version1 branch), things go a bit crazy. If the immediate next command is git checkout master, I get the dread “error: Untracked working tree file ‘vendor/plugins/acts_as_versioned/CHANGELOG’ would be overwritten by merge” error (where the file may be different). Again, this appears to be in a directory where in the version 2 branch it is a submodule, but in the version 1 branch it is not.
Now I’m stuck. Various combinations of git clean don’t seem to help. git status insists the working directory is clean.
So, how do I go back?
If I git checkout master -f, I can switch. But, and a big but, the plugins that aren’t supposed to be in this branch and a variety of files from vendor/rails appear when doing a git status. If I clean those with git clean, I get a whacked directory tree.
Pending figuring this out, I’m switching back to the static inclusion of files in the vendor area and tracking them by hand. I’ll document a solution when found (or if anyone can set me straight or send pointers).
http://blog.buildingwebapps.com/2008/5/20/got-git-submodules-not-a-go-go
I recently ran into this myself. My repository is basically:
I found when I switched from lss (submodules for plugins) to clientX (svn exported code for plugins) I would get a similar error like your:
“error: Untracked working tree file ‘vendor/plugins/acts_as_versioned/CHANGELOG’ would be overwritten by merge”.
What I did to “resolve” it was to delete the plugin folder and then checkout the other branch (clientX). When I came back into the lss branch, I had to rerun `git submodule init` and `git submodule update` to refetch the submodule.
That said, I’m still new to git and I tend to make a backup before removing folders or playing with submodules. Good luck!
Eric
Thanks Eric. I’m still new too, and after years of Subversion, Perforce, and other VCS’, git is really nice for many things. The submodule feature is relatively new, and combine with my inexperience, I’m sure there are some quirks to be worked out someplace.
I’ve rolled back my mixed submodule usage for now, and everything is going fine again. I still haven’t determined the root cause to these issues. I too followed a similar path to your “delete and re-checkout”, but that was tiresome. The init and update routine with submodules is normal, as I understand things, when-ever switching to a branch with submodules.
I’ll post more as I learn it.
-c
Hi Christopher,
I’ve been suffering the same pain have abandoned the submodules approach and switched to braid
Hi Joe, yes, I’ve ended up with a partial solution so far. I’ve been playing with braid, but haven’t made a final commitment to using it. How has it been working for you?
This only happens if you have a folder in one branch say “v1/vendor/rails” and then in the other branch you create a submodule of the same name “v2/vendor/rails” Basically, git is not able to merge the v1/vendor/rails folder with the repository v2/vendor/rails. This will also bite you when you try to checkout an earlier version of the same branch after you add a submodule. I fully expect git to address this soon. This is a common use case and it does not work well currently.
My current solution is that I have a folder git-submodules in which I store all of my plugin submodules. I then manually copy the files from each folder after I update it to vendor/plugins in my project. This means that I have NO submodules in my project so git never complains. This is not a good hack, but it does work. It only takes a few minutes to periodically update the git-submodules folder and copy the contents into my project.
Best of luck!
DrMark