Friday, May 30, 2014

GSoC 2014, Week #2: Slight woes

Hello readers,

This week started with a bumpy start.

Essentially, any horrible thing that could happen with my main desktop, ended up happening. All short from a micro-nuke dropping on the computer.

So that ate up about one and a half days of possible work that I could've put toward layman while I worked on trying to diagnose whether it was a dead CPU or a motherboard issue. I ended up getting a CPU/Motherboard combo to save time diagnosing and ensuring that I'll have a working main desktop by next week. Fingers crossed that nothing else happens and I throw something...

On a brighter note, a lot of work was done this week and I am not really far behind on my timeline at all (thanks to a few days of working longer than usual).

Although the list of what was accomplished isn't as long as last week (for a multitude of reasons) I accomplished what I set out to accomplish.

The following is what I accomplished this week:

1.) Added a re-add overlay command line option:

      In my last post I showed you how to add an overlay using layman, this new command line option will do something similar (hopefully self-explanatory as well).

2.) Added overlay type checking:

       In the event that your repository changes overlay types (ie. from git to cvs) layman will have to handle that. Previously no existing code in layman was set to handle this event. Now, when syncing overlays via the command line option: layman -S or layman -s <overlay>, layman will check the overlay type of the overlay on your local machine and the remote overlay that goes by the same name.

      Under that circumstance, layman will give a warning to the user, notifying them of the change and will then proceed to re-add the overlay with the code provided by the re-add function.

3.) Added overlay URL updating capabilities for all supported repository types:

      Adding the support for this was a fun activity that gave me a little insight on how layman handles different overlay actions as well as the methods that one could take when updating the repository:

      To start, I'll begin by explaining the inheritance of the classes and how this allows them to interact in a neat fashion.

Below, I will depict the inheritance tree for you:

                                     Overlay
                                           |
                                      Source
                                           |
                                   Overlay type

So for example, we want to add my overlay to a local machine with layman (<sarcasm>because I don't do much to the ebuilds it'll ensure longterm stability! </sarcasm>) we will invoke layman as so:

        layman -a twitch153

Hopefully you remember this. Now I'm going to explain in further detail as to how it all works (not entirely, I'm kind of lying to you).

The overlay object will call it's add function, which goes through each available source and attempts to add them by calling source.add():

     def add(self, base):
            res = 1
            first_s = True
            for s in self.sources:
                if not first_s:
                    self.output.info("\nTrying next source of listed sources...", 4)
                try:
                    res = s.add(base)
                    if res == 0:
                        # Worked, throw other sources away
                        self.sources = [s]
                        break
                except Exception as error:
                    self.output.warn(str(error), 4)
                first_s = False
            return res


This then traverses into the source class where you look to see what happens:

    def add(self, base):
        '''Add the overlay.'''

        mdir = path([base, self.parent.name])

        if os.path.exists(mdir):
            self.output.error('Directory ' + mdir +
                ' already exists. Will not overwrite its contents!')
            return False

        os.makedirs(mdir)
        return True


layman will check to see if the path already exists and to prompt the user if it does. So where does the actual action of syncing the overlay come in?

The trick behind it is by looking back at the Overlay class:

    res = s.add(base)

This doesn't really go straight to the source class. In fact, if we were to look at this in a level type scheme, the Overlay class would be level 1, and the Overlay_type class would be your level 2. So what about source? Well, source would be considered your level 1.5.

The overlay_type class inherits all the functions given to it by the Source class:

    class GitOverlay(OverlaySource)

Doing this allows any overlay type to make use of the code in the Source class while also executing its own add() function.

After all of this, what does that have to do with URL updating? Now that you understand the inheritance behind each overlay_type it allows us to make sense of how each overlay type can cleanly have its own updating methods.

The overlay class has a list of available overlay types that allow for a custom method of URL updating, these available types (Bzr, Cvs, Git, Mercurial, and Subversion) all run their own separate functions to update their URLs.

Bzr: To update the repository URL in a Bzr overlay you need to run the bind command:

    bzr bind <source_url>

Cvs: To update the repository URL in a Cvs overlay you need to echo the new repostiory URL into the CVS/Root file of the overlay, and then (if the repository location has changed) you need to replace the old repository location with the new one.

Git:  By running git remote set-url <remote name> <new_url> <old_url> you can update the URL from which git will pull from.

Mercurial: You can update the URL for a mercurial overlay by replacing the old url with the new url in the .hg/hgrc of the overlay. In layman, we make use of sed to do this:

     sed -i 's/oldurl/newurl/' <overlay_location>/.hg/hgrc

Subversion: To update the src-url of an svn overlay you simply need to run svn switch --relocate, followed by the old_url and then the new_url.

For all other overlay types, layman syncs the overlay by grabbing the source url from the installed.xml file in /var/lib/layman/, so when the overlay class recognizes an that an overlay type isn't in the list of supported types, it will simply set the source of the object to one of the new sources reported by the remote database and re-sync the overlay with the newly reported source url.

4.) Polished code already in layman:

A little self explanatory as well, but I cleaned up some of my mess along the way this week as well so you guys can all enjoy clean and bug free (hopefully...) code!

Goals for next week:

  • Continue to refine the code already in layman - This goal will be ongoing throughout the entire GSoC and will never stop throughout my entire life. I will always have to go back and clean up my messes.
  • Add updates to the DTD to add IRC channel information for overlays in the repository.xml.
  • Hopefully add custom config type plugin support (it's being put off...I know).

To anyone interested the source code and my commits can be found on
git.overlays.gentoo.org/gitweb/[1] or on github.com[2] on layman's gsoc2014 branch.


  [1]http://git.overlays.gentoo.org/gitweb/?p=proj/layman.git;a=shortlog;h=refs/heads/gsoc2014
  [2]https://github.com/twitch153/layman/tree/gsoc2014

See you guys next week!

With regards,
    Devan Franchini (twitch153) 

No comments:

Post a Comment