Trac: release management

[Walter Bender]

It'd be nice to summarize these suggestions for improving the Trac
system in a ticket!!

-walter

On Thu, Jun 5, 2008 at 4:25 PM, Garrett Goebel <garrett.goebel at gmail.com> wrote:
> On Wed, Jun 4, 2008 at 3:55 PM, Michael Stone <michael at laptop.org> wrote:
>>
>> If it pleases you, count the number and severity of bugs that have been
>> untouched for longer than 3 months. Then ask yourself where #6454
>> compares fits in to the lists of 'can be addressed quickly' and 'needs
>> to be addressed quickly'.
>
> It depends on what your definition of "addressed" means. Every ticket
> ought to be addressed as in "responded to" within a couple days or a
> week at the latest. #6454 has gone 4 months without any response or
> comment from the ticket's owner or anyone at OLPC/1cc. How many other
> tickets are out there like it?
>
> I'll take you up on your offer for a cloned copy of the Trac db. If
> the data is in there, I'll write you a query which will give all the
> non-closed tickets which have never been changed by the owner.
>
>
>> On Wed, Jun 04, 2008 at 02:50:30PM -0400, Garrett Goebel wrote:
>>> Where is your list of priorities?
>>
>> http://wiki.laptop.org/go/Priorities-2008
>>
>>> How does that map to the list of open Trac tickets?
>>
>> It doesn't. See
>>
>>  http://lists.laptop.org/pipermail/sugar/2008-May/006006.html
>>  http://lists.laptop.org/pipermail/sugar/2008-May/006007.html
>
> Nice general overview of priorities.
>
> There ought to be a way to generate it or something a bit more
> detailed from the Trac database. You'd probably need some way of
> designating a ticket as a meta priority, and then have it block on all
> the tickets required to meet that goal. Speaking of which stuffing
> multiple values into the 'Blocked By' column has a certain odor.
>
> There seems to be some overlap between the Trac fields for Milestone
> and Version. They seem to be being used for some combination of
> 'branch' and 'build'. Whereas where I think you are trying to go,
> would be to have a Release and Schedule. Where 'Release' would be one
> of 08.Autumn, 09.Spring, etc. and would signify that work was to be
> targeted as an update to the branch for that Release and all
> subsequent releases. and 'Schedule' would be the YY-MM you want it to
> land.
>
>
>>> Where do you track the severity/impact of a ticket? I.e. scope of who
>>> is effected
>>
>> Some people try to indicate this information with the 'priority' field
>> on the ticket. In practice, I actually try to skim every change to Trac
>> looking for important issues. Then we discuss them via status summary
>> emails or meetings, like the one we're going to have in 10 minutes at
>> 2:00 PM EST in #olpc-meeting on irc.freenode.org.
>>
>>> Where do you track the difficulty? I.e., general estimate of time
>>> require to address a ticket
>>
>> We don't track it formally; only via discussion and status updates.
>
> Whatever you want to call it, you might find it useful to track the
> scope and complexity of the changes required to fix an issue. Priority
> doesn't get at that. It would allow you to collect historic data which
> could be used to project how much time tickets will take to be
> implemented and how many bug hours you'll get per change.
>
> I understand that the process needs to be as lightweight as possible
> so as not to get in the way of developers actually implementing
> things. But there's the balance to be had with actually being able to
> have historic data available to make possible an efficient use of
> those developers.
>
>
>>> How to you track defects back to the changes (ticket) which introduced them?
>>
>> We don't do so systematically.
>
> It is hard to do if you want to track back to a specific git
> changeset... or Trac ticket.
>
> But you could start to get a handle on it by adding 'Found In', 'Fixed
> In', and 'Tested In' relationships for tickets. How do you currently
> figure out when/where a defect was introduced and fixed?
>
>
>>> How many Full Time Equivalent hours does a given developer represent?
>>
>> A guesstimate: about 25 hrs/wk of coding and 30 hrs/wk of talking for
>> social folks, maybe 30 hrs/wk of coding and 10 hrs/wk of talking for
>> contractors; and 5-8 full days off a month (including weekends).
>
> Is there any list of developers and which slot each fit into?
>
>
>>> What components are the given developers capable of working on?
>>
>> I don't understand this question.
>
> You've got folks who have particular areas of expertise. Or to put it
> the other way, developers who can work in certain areas but not
> others. If your Trac ticket classifies a ticket as belonging to a
> particular area, you can then project how many FTE's you've got on
> hand to work in that area.
>
> I realize that this being an open source project leaves a lot open
> ended. But if you collect the data in a way that you can get at it
> effectively, you can use historic data to verify your assumptions and
> track and make projections against non-employee/non-contractor
> developers as well.
>
>
>>> How long does the assigned developer think the specific ticket will
>>> take to complete? How long did it take?
>>
>> The limiting factors seem to me to be:
>>
>>  a) how long is the critical path of changes necessary to close the
>>     ticket?
>>  b) how overloaded is the required developers?
>>  c) how frequently are the required developers task-switching?
>
> I was ambiguous. What I meant was a).
>
> It'd be nice if there were a field in the ticket for the developer to
> note down how long they think they actually worked on a ticket.
>
> If you combine this with the earlier mentioned field for
> scope/complexity (difficulty) then you can make some projections on
> how many FTE outstanding ticket hours you've got based on historic
> data. And you can assign each 'difficulty' level an average time to
> completion based on a running aggregate of the historic data.
>
> You can also make some projections on the average number of defect
> hours you can expect will eventually be entered based on how the
> ticket is classified.
>
> Put it together with allowing tickets to have a 'schedule' where
> schedule is YY-MM (ex: 08-07) And you can hope to create a realistic
> schedule of what you hope to accomplish month by month.
>
>
>> Also, you've got to be careful here to specify a ticket workflow. I do
>> release work at the moment, so I don't consider issues to be "fixed"
>> until they're fixed _in a released stable build_.
>
> true.
>
> I'm assuming by fixed you mean implemented and tested.
>
>
>>> How long must a ticket sit dormant before it gets bumped and someone
>>> takes notice?
>>
>> It's not a matter of taking notice. It's a matter of being reminded at a
>> time when there's nothing more pressing on the priority queue.
>
> I'm going to assume that we both agree that some action should
> initially take place. Perhaps we don't. I think the person who filed
> it should be able to see within a few days whether or not the ticket
> is accepted or rejected, who it has been assigned to and how it has
> been prioritized.
>
> Define:
> o  who should be reminded
> o  'nothing more pressing'
> o  'priority queue'
>
> And then we can see if it can be automated...
>
> I'm probably not the only person who has ignored things in my queues
> because I'd rather work on the interesting problems or the low hanging
> fruit.
>
>
>> (Most developers seem to have actual work queues which are about 5
>> tickets long. In practice, their Trac ownership lists are often 30-100
>> tickets long. Go figure.)
>
> I don't understand the difference between an 'actual work queue' and
> 'ownership'. Can you explain?
>
>
>>> What is your rate of defects per change? How does that break down by
>>> severity and difficulty?
>>
>> Are you measuring by source commits, packages, test builds, candidate
>> builds, or releases?
>
> Trac tickets.  Source commits might be better. Some analysis of the
> composition of git changesets associated with a Trac ticket would be
> better. But in practice Trac tickets with fields for complexity and
> length of work required are probably just as good and a whole lot less
> complicated.
>
>
>>> Are tickets tested? Tested by someone other than the implementer?
>>
>> Sometimes. Also sometimes.
>
> Is that what 'verified' indicates? How do you know what branch/build's
> it was tested in?
>
>
>>> Are tickets reviewed before being closed?  By someone other than the
>>> implementer. Who?
>>
>> See #7014 for an example of the problem.
>
> Looks like the big problems are easier to solve if you identify them
> as a bunch of little problems problem. The ticket should probably have
> been broken down into lots of smaller tickets and then updated to list
> them as its blockers. There's mention of other tickets all
> throughout... but you're not using the 'Blocked By' or 'Blocking'
> fields...
>
>
>>> You _talk_ about priorities and wise time management but certainly
>>> haven't done a good job communicating how you plan to go about doing
>>> it.
>>
>> What should he do instead? (Keep in mind that Peru is receiving
>> something like 40k additional laptops next week that need new software
>> which Scott is the only person working on full-time. [He's both the
>> release engineer and he's fixing and testing individual issues as
>> documented by his recent changes to Trac.])
>
> This comment was taken out of context. It was in response to a mild
> rant from Scott in which in addition to stating that he was overworked
> and needed to prioritize... mis-characterized my original email as a
> loudly insisting that my person problems be worked on *right now*.
>
> None of which answered the original question as to how the OLPC
> process can be so broken as to allow a newly filed tickets to be
> completely ignored forever.
>
> I wouldn't pretend to know what Scott should be doing with his time. I
> was only asking that the OLPC improve its process so it does not waste
> mine.
>
>
>>> Besides, how can you hope to prioritize if you don't enumerate your
>>> resources,
>>
>>  http://wiki.laptop.org/go/Available_Labor-2008
>
> Since (o) employee/contract is grouped... it might be useful to break
> them out. If you and I are both interested in my working on your issue
> tracking system to make it more useful for resource planning.
>
>
>> Also notice how I'm splitting release prioritization from development
>> prioritization into separate management problems.
>
> No I didn't notice. I don't see the words 'release' or 'development'
> mentioned on that URL.
>
> It would be interesting if there were a mapping from how you've
> categorized these people to how Trac tickets are categorized. I was
> assuming the 'component' field could be used for that purposed.
>
>
>>> constraints, and interdependencies? How can you balance
>>> work queues if you can't quantify them?
>>
>>> How can we as outsiders expect our interactions with the OLPC to be
>>> addressed in a timely manner?
>>
>> By bartering for the time of the people whose help you need.
>
> I can't even contact the owner of ticket #6454. Apparently, I'm
> supposed to go on IRC, hang out and hope to catch him.
>
> Barter with what? My overly polite and friendly disposition :-) ?
>
>
>>> Even if addressed only means being notified that the given issue won't
>>> be addressed.
>>
>> Maybe we should put it in big bold letters on the front of Trac, but if
>> your ticket hasn't been commented on within a week, that means that no
>> one is currently watching.
>
> That much is obvious. The question is how you get a hold of someone to
> find out if they ever intend to do anything about it.
>
>
>>> How do I create a report? http://dev.laptop.org/wiki/TracReports tells
>>> you about reports, but not how to create one...
>>
>> Sadly, you need to be a Trac admin to create reports. What would work
>> better would be to work with a local copy of the Trac DB. We could
>> probably set up a cron job to make one if it would help.
>
> That would be nice. Should I open a ticket?
>
>
>>> You folks are always asking people to open tickets. But how many of
>>> those tickets result in n-way communication? How many open tickets are
>>> out there that go unanswered? 610 have "Never Assigned" for their
>>> Milestone. Does the OLPC have anyone out there tasked to track down
>>> abandoned tickets? Why should I invest my time?
>>
>> OLPC has no one tasked to track down abandoned tickets. Who should they
>> assign?
>
> Who ever assigns tickets when tickets need to be assigned. Would that be you?
>
> Make a sql query that does the work for you. Give me that copy of the
> Trac database and I'll take a swing at doing it for you.
>
> cheers,
>
> Garrett
> _______________________________________________
> Devel mailing list
> Devel at lists.laptop.org
> http://lists.laptop.org/listinfo/devel
>