Merging of the Mozootle branch into the trunk

• Mozootle branch
• Verbatim project
  • Meetings
• Merging work (TODO)

• Dwayne has committed fixes to the mozootle branch to ensure that it works correctly with SQLite and without LDAP.
  • Status: complete and ongoing
  • Notes: did not attempt to downgrade the SQLalchemy version.

We will be using git to perform the merge of the Mozilla Pootle branch with the Pootle trunk. The flexibility of git will allow us to replay commits that are incorrectly merged.

Some things will need to be realigned, probably before the merge to create the optimal situation for the merge.

• “po” vs “projects” directory mentioned below
• The most recent changes committed to trunk (tests, kid, cache improvements, poheader awareness)

• Step through the commits, looking for candidates to cherry-pick,
• Cherry-pick the candidates, fixing conflicts as they arise,
  • some of the conflict resolutions will by necessity be quick fixes (given the volume of code to merge),
  • conflict resolutions which cause problems can be fixed and replayed,

FW: Won't it be easier to merge everything and remove stuff that we can't merge?

• Styling
  • Must fix: Language specific styling gone - needs to be ported back from trunk. Trunk has CSS that only applies to specific languages. For example disabling underlining for Vietnamese hyperlinks, choosing specific fonts for Persian (all at the top of pootle.css)
  • Search box styling leaving it a bit in the air
  • translatepage weirdly spaced - space on either side unused, but no space for terminology or TM (clouserw: the styling for this page isn't finished yet. If it works now I'd like to punt until the switch)
• Hyperlinks broken in footer - clouserw, r8955
• UI strings changed causing untranslated strings. (clouserw: I'd suggest not even doing these until we finish the UI changes. I understand you want to release though so your call. We'll try to keep changes minimal.)
• ajax suggest interface different from real editing interface (special chars, size) (clouserw: the ajax interface is designed to be a quick way to suggest - not full blown editing. the “size” user preference is outdated and I'd suggest not having it when we migrate.) (friedel: I think the issue is more that the size of the suggestion box is too small for anything but the smallest units - autoresize? Have to confirm). The issue is probably more of a fundamental one. Do we want an editing interface without the helpful features that Pootle offers? Can we easily provide these with the AJAX interface?
• ajax suggest interface not working for plurals
• RTL issues (seems to need complete rework)
  • broken on translatepage
  • Progress summary in stats tables
  • Header/banner
  • HTML dir attributes not correctly specified on when viewing suggestions

• Must fix: plurals seem untested - fixed some. Changes to translatepage::maketable weird. Possible to review these?
• Must fix: Interface language not detected from http Accept-Lang
• Must fix: Authentication broken unless python ldab bindings are installed - do we want this? There is/was code using the ldap library even if it couldn't be improted.
• Must fix: users.py:PootleSession:validate is redefined - second one is new, the first is also changed in mozootle
• Can't create a user from admin interface - fixed by AA
• Can't activate a user when creating account (clouserw: is this a regression? the admin tools are going to get a big rewrite; I'd suggest not a blocker) (friedel: Agreed)
• images vs img - fixed by WW
• test scaffolding not close to what we need almost there now
• Request cache kept but disabled with comment ”# Disabled per mozilla bug 459005” - clouserw, r8956. This didn't work for a while for me. I talked to winterstream on irc and I think he fixed it a long time ago. Regardless, it passes my test case now.

• Creation of a script to upgrade user information from the old flat file format to the database system introduced in the Mozilla branch,
• Refactoring of the database related functionality (statsdb_mysql.py and statsdb_sqlite.py) into a unified system,
• Refactoring of the authentication code so that we can support OpenID in addition to LDAP and the database-based authentication system.
• Modify the toolkit statsdb so that it can be initialized with a DBAPI 2 connection. In this way, the stats data can be stored in any database with the DBAPI 2 connection (for example, in Pootle's database). This implies that the stats code must use lowest common denominator SQL.
• Fix the statsdb code so that it will erase stats data that is incompatible with the current code. This happens if we change the layout of tables, for example.
• Provide migration tool from prefs to whatever. Possible start by paulproteus

• Changing the name of the projects directory from “po” to “projects”.
  • Needed change for Mozootle?
    • Nope, we don't care. Just need to decide. --clouserw
      • Ok, let's stick with “po” to follow the GNU convention and to simplify the merge (hopefully)
• Various UI changes that we want to review for inclusion.
  • Translate page is very wide - no place for “related” (TM and terminology)
    • yep, this needs to be redone after django lands -- clouserw
      • Since the UI needs to be reviewed entirely anyway, let's not block on this and fix it as we review everything.
• How much of the branding changes should/can we merge?
  • Many of these are isolated in the html/img directory. Some changes are merged into pootle.css
    • The only thing specifically mozilla is the logo which can be swapped easily
      • Part of the UI review - no blocking necessary
• New stats view and activity summaries - suitable for all projects?
  • We need to take to the mailing list
• How to handle the dependency on unreleased versions of SQLAlchemy (this will likely resolve itself). Also see notes below about ORM.
  • As soon as things are working, we can look at moving to something else as discussed below. In other words, let's not block on this one.

• The toolkit cannot depend on Django's ORM. So either it must use SQLAlchemy, a hand-rolled ORM solution or well written, platform neutral SQL.
• It would be good for Pootle to use Django's ORM for non-toolkit data. This means that we can use Django's myriad modules to help with things like sessions and authentication. It's also useful for a savvy Pootle admin to be able to modify something like user data via Django's admin interface (which is only available if one uses Django's ORM).
• Add the ability to initialize the toolkit's stats with a Django's DB connection (which acts like a DBAPI 2 connection). This allows toolkit data to be stored in the same database as Pootle data, without imposing a Django dependency on the toolkit.
• Unify statsdb_mysql and statsdb_sqlite into statsdb. Preferably we can push as much as possible down into the toolkit.

* The use of kid templates in Django is documented here.

* Using sqlalchemy with Django

• (Since we're not using much sqlalchemy right now (and what we are using is dependent on an unreleased version) I'd rather just use django's ORM --clouserw)

1. Styling changes (r7537, r7650, r7680, r7681, r7682, r7683, r7684, r7689, r7729).
2. Fixes to RTL support (r7730). Needs to be tested thoroughly.
3. The addition of LDAP. Authentication occurs either via LDAP or by consulting the users database. This is based on a fallback mechanism (first LDAP is tried, then the database). See users.py. (introduced in r7679).
4. Changes to Pootle were frequently merged in (r7688).
5. Changed back button behaviour to do a GET instead of using the browser's history.back() function. (r7690).
6. Fixed permission bugs and concepts (r7693, r7706, r7709, r7711, r7725).
7. Encode the user's language in the URL. This code continually redirects if it can't read user information from the database (needs to be fixed). (r7694, r7731).

This list is far from done.