<div>Just a quick response with first impressions about your comments.</div>
<div> </div>
<div>There was one productive misunderstanding in what you said: I was thinking of (well, actually, I'm almost done with a first mockup using IDLE) doing ALL the translation in a layer that was invisible to the interpreter. Your idea of having a block of assignments in a module's global namespace may be useful - although it opens up the whole can-of-worms of namespace and scope. I'll have to think about it to see if it's workable.
</div>
<div> </div>
<div>Further responses:<br><br> </div>
<div><span class="gmail_quote">On 8/13/07, <b class="gmail_sendername">Marc-Antoine Parent</b> <<a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:maparent@gmail.com" target="_blank">maparent@gmail.com
</a>> wrote:</span> </div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">However, Jameson, if I may, I would take issue with a few assumptions<br>of your model: most especially that of a "preferred language" for
<br>modules.<br>I am referring to your point 3:<br>> 3-This dictionary ONLY contains translations for the "public<br>> interface" of somemodule.py, that is, those identifiers which are<br>> used in importer modules. It also defines a single, unchanging
<br>> "preferred language" for that file, which is the assumed language<br>> for all non-translated identifiers in that file.<br><br>I am especially interested in collaborative work; and I believe it is<br>
not unreasonable to hope that children between schools in different<br>countries will get to share some work.<br>That would mean that a given modules may have many editors, possibly<br>introducing identifiers in more than one non-English language.
<br>From that point of view, "preferred language" is a feature of an<br>editing environment, not of a module. New identifiers should be<br>individually tagged by language; I see that tagging as appropriate<br>work for the editing environment. Basically, upon loading the file,
<br>all local identifiers would be read in memory; upon saving, new ones<br>would be saved with a language tag. (Plausibly as a postfix,<br>Identifier_i18n_2letterLanguageCode...)<br>(I would otherwise follow Mike's suggestion to use a fixed
<br>transliteration table for non-latin scripts.)</blockquote>
<div> </div>
<div>I originially intended a design something like this. Here are the problems I found:</div>
<div>1. requires an assumption that all those who edit a given module have our magic editor, and that they all have their "preferred language" set correctly. (Imagine a Belizian child who left it set on their country's default "English" but actually edited in Spanish, Garifuna, Creole, or Qeqchi).
</div>
<div>2. It also means that the dictionary for a given module could get filled up with translations of each functions internal variables, etc. </div>
<div>3. I see no way of parsing a file to see what is "public" (for importing) and what is "private" (module internal). Thus the entire dictionary for a module would have to be imported with the module.
</div>
<div>4. For similar reasons, modules would have to import the dictionaries for modules 2 levels up in the import inheritance. My paradigm lets you manually do this only for the rare cases it's needed.</div>
<div> </div>
<div>And what is the benefit? Modules which are a "language soup", maintained by an international coalition of children who can't even talk to one another.</div>
<div> </div>
<div>I support the idea of international, cross-language programming collaboration. However, I think that a basic assumption would be that there were at least one common language of communication between the participants, and that any given module has an owner and thus a preferred language for all its still-untranslated identifiers. If somebody wanted to "add to" that module, they'd have to either write in that preferred language or use an explicit import and subclass (or, of course, just do it in their language for their own private use, because that file would then be explicitly marked as "messy and suggested not for sharing")...
</div>
<div> </div>
<div>Do you still object? Because, um, I hate to be gauche, but well... I'm doing the coding here so far... so unless I hear a consensus that I'm doing it wrong I think I'll continue with this assumption.</div>
<div><br> </div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">.....<br>Even if someone decides on a better translation later on, more than <br>one version may be kept in the translation block.
</blockquote>
<div> </div>
<div>You just blew my mind. I need to think this suggestion over with a pencil and paper.</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">This has the disadvantage of polluting the code, but the advantage of<br>polluting the filesystem less.<br>
<br>I am realizing a broader application of this mechanism: the <br>translation block could be tagged with a revision number (if<br>__revision__<540:), and the "import" command could mention the last<br>known revision; so translation blocks would only be activated at
<br>need. But that's all another story. </blockquote>
<div> </div>
<div>Again, I have to think about that.</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Another quick related note: What if someone adds a translation<br>between two non-English languages? In your first email, you
<br>explicitly forbid it; I am not sure that is necessary. (I am not sure<br>you think of it as necessary in your later design as well.) Clearly, <br>however, X to Y translations may have to refer to the history (as<br>language X is replaced by English) so as to become English to Y.
</blockquote>
<div> </div>
<div>I think that in this case, we could make an explicit English placeholder, along the lines of "fr_une_fonction_in_English". Then when the English is added later, our magic can clean things up.</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">To finish with your design points, you introduce what I see as a<br>severe limitation in your point 4: <br>
<br>> 4-There is good UI support for creating a new translation for a<br>> word. However, the assumed user model is that words will be<br>> translated INTO a users preferred language; FROM the context of an<br>> importer module (you'd generally not add translations for a module
<br>> from that module itself, since generally you wouldn't even have<br>> modules open whose preferred language is not your own); and<br>> therefore WITH an explicit user decision as to which module this<br>
> translation belongs in (they want to use their language for<br>> identifier X which is in English, well, they must have had a reason<br>> to write it in English rather than their language so they <br>> presumably know what imported module it comes from.
<br>What really made me jump is the notion that "you wouldn't even have<br>modules open whose preferred language is not your own". Again, this <br>assumes a single preferred language per module, which is something I
<br>would rather avoid, and I believe is not necessary if identifiers<br>have a language identifier.</blockquote>
<div> </div>
<div>If you have a good solution for the problems I mention above, I'd be happy to consider it. As it is, I'm not saying it's impossible, but my feeling as I tried to code it initially was that if you try to make things too "magic", eventually you're going to make the wrong assumption and you're going to create some bugs in the code being edited that are really really hard to track down. I'd rather keep things "simple" (my current version of the translator module is over 200 LoC, by the time I add file IO for the translation dictionaries and clean things up, I expect it will be around 300 plus docstrings) so that it is at least possible for a programmer who hasn't studied the code to have an intuitive grasp of what's going on "under the hood" in their translator.
<br> </div>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">However, I suspect your mention of "from the context of an importer <br>module" comes from the issues you encountered with memorizing the
<br>import structure. I would like to hear more about the problems you<br>ran into there, because I believe it is necessary (for reasons to be<br>detailed below.) </blockquote>
<div> </div>
<div>The "from the context of" is more a UI than a logical consideration. It is a way to encourage people to translate where it's most natural to do so: INTO languages they actually use, and WHEN it's actually useful to do so, and also WHEN they would naturally be conscious of what file the translation is native for. It would be possible to do translations in other contexts, but the special UI sugar (context menus, tooltips, and the like) would encourage you to do it in this context.
</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Now, a few suggestions and pitfalls of my own:<br><br>a) I believe there should be one translation file per language. More
<br>file pollution, less parsing.</blockquote>
<div> </div>
<div>My current mockup is going to use csv for its dictionary file format - Engish in the first column and one language per additional column, in order of addition. Internally it is just a 2d array - a list of lists. A "row" is only as long as it has to be and can have any number of "empty" values as long as at least one value is defined. The parsing on this file is, needless to say, trivial. It would not be much harder if you converted it to XML, with one word in one language per line, and you'd get more-fine-grained diffs for free.
</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">I suspect that something akin to the getinfo file structure would be <br>appropriate:<br>package/module1.py
<br>would be translated in <br>package/_t9n_/fr/module1.pyt<br>package/_t9n_/fr/module1.pyto (object, like a .mo file)<br>package/_t9n_/es/module1.pyt<br>package/_t9n_/es/module1.pyto<br>and so on.</blockquote>
<div> </div>
<div>OK, that is a valid point, it might be nice to follow getinfo format to leverage existing tools. However, it is very useful in my model that once a word exists in the dict for any language, it is noticable from the perspective of all languages, even if it has not been translated at all yet. This is a benefit of a single multilanguage dict.
</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">b) A particularly fancy editor would color-code words in other <br>languages instead of showing the _i18n_xx tag.
<br>Of course there would be a way to access online translation services<br>to get suggestions (as has been suggested by many.) </blockquote>
<div> </div>
<div>yes.</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">c) Sci-fi scenario: any new translation suggestion by a child or <br>educator should be made available to others using a distributed
<br>database system... (they are likely to work on common projects, and<br>hence on common modules.) <br>The children educators known to be knowledgeable about a given <br>language pair should have a way to vet translations in that database.
<br>Oh, and let's send it to planet python so we have a basis to build<br>the translation files to the standard library for very obscure <br>languages ;-) <br>(OK, that _is_ sci-fi. Still worth thinking about!)</blockquote>
<div> </div>
<div>I've been thinking about it. The problem, as you point out, is hygiene; you need to build some whole new ratings/trust model (and UI) for conflicting translations. Still, I think that if you make a clean design for one person, adding this functionality later should actually not be that sci-fi/hard. This would certainly give a "critical mass" to the concept!
</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">d) Back to earth: I said we really had to know the import<br>structure... here is a slew of related problems:
<br><br>Suppose we are editing a module that is importing something from the <br>core library:<br><br>from moduleX import f1<br>from moduleZ import f2<br>f1()<br>f2()<br><br>Now, suppose f1 and f2 both translate as "sigma" in the current
<br>editing language... Then, though the .py code is unambiguous, the <br>translated on-screen code looks ambiguous; and worse, the un-<br>translation process on save is not well-defined.<br>The solution is to actually un-specify the imports in the source code:
<br><br>import moduleX<br>import moduleY <br>moduleX.sigma()<br>moduleY.sigma()<br><br>This refactoring should be possible in most cases, unless two top-<br>level modules have similar translations. (say moduleX and module Y
<br>both translate as "modula") <br>This situation should be marked as an error; or alternately _display_<br>the following:<br><br>import moduleX__i18n_en_ as modula_1<br>import moduleY__i18n_en_ as modula_2<br>
modula_1.sigma()<br>module_2.sigma()<br><br>This is not an interpreter-level change, but a disguised display. (or<br>rather a refactoring which can be memorized, and reverted by the<br>untranslation machinery.)<br>Note that display-only import disambiguation may also be necessary if
<br>the above code happens in a core library file (which we would never<br>modify.)</blockquote>
<div> </div>
<div>I have already implemented display-only disambiguation (and reambiguation - consider when one English word has different French translations in moduleX and moduleY, you need to call it moduleX___voir___moduleY___ve , sorry my French conjugations have been overloaded by my Spanish ones so that may be wrong but you get the idea). I don't see that on-disk refactoring is necessary, though the UI should encourage/support you to manually disambig your translations.
</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">In any case it is useful to flag as an error any translation that<br>introduces ambiguity within the same namespace.
</blockquote>
<div> </div>
<div>Again, UI warnings or refusals to create bad situations, but sensible handling/resolution support when they occur anyway.</div>
<p>I have found one case where you actually need to do a manual disambig for every case of an identifier in your file. Say you are importing an English color_module with the translation red -> rojo. You're also importing an UNTRANSLATED spanish_networking_module which uses the identifier red. Now you want to add the translation network -> red to spanish_networking_module. If you were working in Spanish, the editor could have warned you when you typed "red" and helped you change it into "es___untranslated___red" (using in-scope interpreter-based refactoring to save this as something other than "red" in on-disk). If you were working in English, you could at least have had color cues that would let you distinguish between "spanish red"s and "english red"s as you coded. But if you had ignored the color cues or had been working in a non-translation-aware editor, you now have to go through the file and tell the editor whether each example of "red" is English or Spanish.
</p>
<p>This is the kind of mess which I want to handle right, of course, but also to keep to a minimum. With my use model, I think this will be an extremely rare case (and, as I said, there will be UI that will let an aware user fix it ahead of time). I strongly suspect that if we abandon the per-file preferred-language model, this kind of mess will be common.
<br></p>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Similar transformations may be made necessary by "from moduleX import <br>*" syntax.<br><br>None of this is simple, as I said; but alas probably necessary.
<br><br>e) Would we display numbers as the equivalent numerics in other<br>writing systems?</blockquote>
<div> </div>
<div>I was unaware that there was any place in the world where first numeracy was not in arabic numerals? I know that there are numeral characters in Asian languages, but I thought that math was taught in Arabic numerals even there - just as people here in Guatemala learn base-20 Mayan numerals but don't use them day-to-day, even a native Mayan speaker who doesn't speak Spanish will only speak in Mayan numerals up to at most 19, beyond that they use Spanish with Mayanized phonetics.
</div>
<div> </div>
<div>Is this important and worth doing? Shouldn't be too hard if it is.</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">f) Docstrings... are another issue entirely. I still like my idea of <br>a distributed database, so children puzzling out a foreign (to them)
<br>docstring with online help can put their minds together.</blockquote>
<div> </div>
<div>Absolutely. Let's say: docstrings are our strong motivation to get around to the version-2-with-distributed-database.</div><br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">OK, I am giving more problems than solutions, here; and<br>unfortunately, my spare time is otherwise quite occupied, so I doubt
<br>I can contribute to implementation; still, I hope that spelling some<br>of these things out is useful to others. I'll try to keep my thinking<br>cap on as this discussion evolves.<br><br>Cheers,<br>Marc-Antoine Parent
<br><a onclick="return top.js.OpenExtLink(window,event,this)" href="http://maparent.ca/" target="_blank">http://maparent.ca/</a><br><br>P.S. I _love_ your idea of arrows in the margin to indicate flow!<br></blockquote>
<div>Thanks. I think it would be good too, but this is one I do not plan to program myself.</div>
<div> </div>
<div>Hasta luego,</div>
<div>Jameson<br> </div>