-
When gucharmap starts up, it always shows the Arabic script block because that one is the first script in the list. I just cooked a little patch to make it show the script block for your language at startup.
Look! Letters I can read! -
API requests that fell through the cracks:
Which radio button is selected? Mea culpa for forgetting about implementing it for GTK+ 2.8, since I had ranted about it in the past.
Accessors for GtkDialog.vbox and GtkDialog.action_area. Language bindings have to write glue code by hand right now. (Also, there's a really old, very likely obsolete list of missing accessors.)
Convenience API to register stock icons. Right now you have to go through GtkIconFactory, which is rather complex. This is the only example I could find of how to register stock icons by hand.
gdk_pixbuf_new_from_in_memory_data(); currently you have to feed your memory buffer to a GdkPixbufLoader. Even the relatively new gtk_selection_data_get_pixbuf() uses the loader trick.
There are 273 requests for API additions in GTK+. Some of them are quite small. Which ones do you care about to make a patch?
-
Profiling with sysprof
Today I committed the start of a profiling framework for GTK+. Right now it is very simple; it creates a window, shows it, waits for it to be fully exposed, and destroys it. It does that N times and prints the timings to stderr. The idea is to run that under a profiler to see what's going on.
Søren's sysprof so far has been great. Installing it is a snap: you just compile it, run modprobe sysprof-module and run the sysprof GUI. Sysprof is available from the GNOME CVS repository in the sysprof module.
It is not clear what people mean when they say "GTK+ feels sluggish". For this, I want to find out two things. First, how long does it take to set up a typical window? Second, how long does it take to update the display when the user does an operation?
"Set up a typical window" is a terribly under-specified problem, of course. A typical window for GNOME can be the main window of an application (menu bar, tool bar, status bar, content area), or a dialog box (layout-ish widgets like boxes and tables, plus entries, treeviews, buttons...).
"Update the display when the user does an operation" is also under-specified: which operation? I guess I'll start with "right-clicking to bring up a menu is slow", or "menus are slow to pop up" in general. One can later test things like scrolling a GtkTextView or a GtkTreeView, which in turn depend on what data is loaded into them.
My little testing program for GTK+ creates this window:
It has a menu, a toolbar, a busy content area with several panes with trees and text widgets, and a status bar. I'm interested in how long it takes to set up such a window, how long it takes to prepare for painting it for the first time (realizing the widgets and mapping them), and finally, how long it takes to paint the thing the first time.
Soon I want to test how long it takes to repaint the whole window (say, when you flip virtual desktops) once the window is up and running on the screen. But not yet — the testing framework needs code to be able to do that.
So, on to the first results, courtesy of sysprof:
The three stages of initialization:
So main() takes 46.34% of the system-wide time. My machine was doing nothing else but running the test program (aside from running an idle GNOME desktop), so perhaps that 46.34% value should be scaled to be 100% minus the time spent in the X server.
Within main(), and my timer_time_widget() utility, there are three sections:
-
19.81% - gtk_main(): This should amount to exposing the widgets for the first time. The widgets only get exposed exposed once, and the window is destroyed immediately afterwards.
-
14.44% - gtk_widget_show_all(): This is realizing and mapping the widgets. The style stuff kicks in, icons get loaded.
-
11.99% - create_cb(): This is creating the widgets.
Looking for hot spots:
Sysprof, like its cousin memprof, has a three-pane display:
-
A list of all the profiled functions. The default sort is descending based on cumulative time. So at the top of the list you'll see very generic functions like main() and g_signal_emit(). This does not mean that g_signal_emit() is taking all your time; it means that the signal handlers do most of the interesting work in your application. Scrolling down the list you'll find specific functions, which is what you actually want to optimize.
-
A list of callers. When you click on a function in pane 1, pane 2 will display the callers for the function you clicked. For example, if you click on g_signal_emit() in pane 1, then pane 2 will display its callers: gtk_widget_event_internal(), gtk_widget_show(), etc. Once you have located a busy function in pane 1, you can use pane 2 to see which path of invocation is taking the most time.
-
A tree of function calls. When you double-click on a function in any of the three panes, sysprof will fill pane 3 with all the sub-functions that your function called.
What's using your time in GTK+ apps:
Scrolling down the list in pane 1, past the generic functions like main() and g_signal_emit(), we begin to find interesting stuff.
The first function in the list that is not totally generic is pango_layout_get_extents_internal(), at 12.32% of the system-wide time:
There are no surprises here: a good chunk of GTK+'s time is spent measuring text. Look two rows above it; gtk_widget_size_request() accounts for 12.32% of the system-wide time, or about 1/4 of the time for the test application alone.
It's not entirely obvious how that process can be sped up. The time of the child functions of pango_layout_get_extents_internal() is rather evenly spread out between splitting the text into LTR/RTL extents, loading fonts, looking up glyphs... There's already a pretty heated bug about this, with no obvious conclusions.
Alternatively, can we avoid measuring so much text altogether?
Going down the list in pane 1, we find gtk_text_layout_get_line_display(), which spends 6.26% of the system-wide measuring text, and gtk_tree_view_size_request(). which spends 3.55% of the time doing the same thing.
So, no surprises there, either: "widgets which display a lot of text spend a lot of time measuring text".
Going down the list in pane 1 again, we find that gtk_ui_manager_real_get_widget() accounts for 6.34% of the system-wide time. This is the function that creates menu items and toolbar buttons. The time inside it is more or less evenly spread out among fetching icons, creating widgets, adding widgets to menu shells, etc.
The next two big ones are gtk_tree_view_expose() at 6.17%, and gtk_text_view_validate_onscreen() at 5.87%. "The complex widgets are the most expensive to redraw"...
Conclusion
Uh, no conclusion for now, other than we have a lot of work to do.
Still, I'm delighted that sysprof makes it actually pleasant to browse profiling logs.
-
-
What I would like to have in sysprof:
A way to list all the parent callers of a function, even when they are not direct callers. This would let me know some of the code paths in GtkTreeView which do measurement of text — that may indicate places which need different algorithms, or caches, or something else. For example, I don't know if cell renderers are measuring text more often than they need.
A treemap display of the cumulative times like in Kcachegrind.
-
My latest rant on distros who have to maintain old versions of GNOME, the QA problems of this, and why old bugs are hard to fix:
Bugs
I work in the desktop team at Novell, and a large part of my work consists of maintaining NLD 9, which uses GNOME 2.6. When a bug comes in for that version, my life becomes a little hell of trawling old bug reports in bugzilla.gnome.org, correlating them with CVS commits (and Bonsai doesn't work), asking my friends who work for other distros whether they have fixes for the bug already... This kind of work leaves me little time to play with HEAD and test it.
Examples: I routinely ask JRB if he knows whether Red Hat has a fix for a bug I got assigned within Novell. Glynn saved my ass with some SMB patches for gnome-vfs.
My ideal bug fix in Bugzilla is like this: it has an attachment with the patch; it has a comment with the corresponding ChangeLog entry that went into CVS. Then I can grep the ChangeLog out of CVS for the text in the bug, get the date, look at commits for that date, find all the files that got changed with that commit, etc.
Mark Shuttleworth, during his keynote at GUADEC, gave an awesome demo of Ubuntu's meta-bug tracker: they maintain pointers for the same bug across the different bug trackers of different distros, and thus they magically know when any of them manages to fix the bug. Everyone would save a lot of time if something like this were a public resource.
Bugs keep coming in even for old stable versions. Novell is about to release a product that works on top of NLD 9 (GNOME 2.6), and we are finding interesting and hard bugs in there, mainly in gnome-vfs and Nautilus, that were still present in GNOME 2.10 and HEAD. If distros fix these and take the time to propagate the fix to newer versions, everyone benefits!
Getting fixes for old versions of GNOME
Patches for old versions are traded in the black market. You have friends in another distro? You ask them first, "did you guys already fix this?" Those patches don't ever manage to reach CVS, where everyone would be able to get them.
Generic fixes don't get committed to old stable branches in cvs.gnome.org — why bother? People only commit them to the latest stable branch and to HEAD, if the bug is relevant there. Other distros then have to do the same fix-searching actions that I mentioned above, and everyone wastes time.
Example: Bill routinely prompts me for a11y-related fixes to GtkFileChooser in GTK+ 2.4 (paired with GNOME 2.6, from NLD 9); I point him to the Novell SRPM which has those fixes.
Fixes in newer versions are often hard to backport because many things have changed. The fix may be a one-liner in newer versions, but it also depends on tremendous changes to the underlying infrastructure.
What if we made old stable branches on cvs.gnome.org free-for-all? Also, what if we encouraged distros to commit their patches directly to old stable branches? This would benefit everyone who has to use an older stable branch; it would also reduce the number of questions to the poor module maintainers, who don't have time to monitor what people have done to old versions about which they no longer care.
The "almost-latest" syndrome
GNOME development is going on. The latest stable tarball for a module is foomodule-2.8.7.tar.gz; the following branches are maintained:
foomodule-2-8
HEADAt that point, GNOME does its next major release. Foomodule branches:
foomodule-2-10
HEADNo one ever does a last tarball for foomodule-2.8.8.tar.gz. Thus, the "last" fixes in the foomodule-2-8 branch are effectively lost, since distributors can only package the latest tarball.
All distros have packages with a "update-to-latest-in-branch.diff" patch. If that last tarball existed, distros would save a lot of time.
QA
Each GNOME version is hard to re-package for distributors. Old patches no longer apply; large parts of the infrastructure have changed, requiring rethinking of the distro's approach to fixing things.
Distributors can't make their QA teams test things until they finish the packaging work.
Expect flurry of incoming bugs once QA gets their hands on the packages: this is the pattern at Novell and I'm sure other distros as well.
Lack of automated tests makes for a shitload of duplicated QA work. Novell is working on LDTP! This should give us automated tests that will benefit everyone.
GNOME-wide issues
We still allow new APIs to come in undocumented. How do we expect upper layers of the platform to use them? If I were in the release team, I'd say "no go" for libraries which introduce new APIs without documentation. This means both our beloved gtk-doc inline comments *AND* examples.
-
Starting up a GtkFileChooser is slow. On my 1.7 GHz laptop, a file chooser never seems to come up in under a second. That's way too long, given that even the old Midnight Commander could display a large directory in under 0.3 sec.
Basically, I want to reduce the startup time for a GtkFileChooser. This is not the same as making it fast for large directories. Setting up a file chooser involves a number of things: loading the dynamic library for the VFS backend, initializing the MIME code, reading icons for the first time...
Yesterday I added some logging code to get timestamps when interesting stuff happens during the file chooser's startup. Taking a breakdown of the log, this is how much time each step takes in seconds:
Step Time gtk_file_chooser_dialog_new() 1. Initialization 0.999188 2. Append home, desktop 0.229314 3. Append volumes 0.004287 4. Append bookmarks 0.136405 5. Set to cwd 0.036381 6. Style stuff 0.142345 gtk_file_chooser_set_current_folder() 7. Set to right directory 0.033232 gtk_widget_show() 8. Reload folder 0.057892 9. Reload bookmarks 0.030040 Total 1.756176 Steps 1 through 6 happen inside gtk_file_chooser_dialog_new(). Steps 8 and 9 happen inside gtk_widget_show(chooser).
-
Initialization. This is the scary part: all of this comes from dynamically loading the VFS backend. That's essentially _gtk_file_system_create(), which loads $(libdir)/gtk-2.0/2.4.0/filesystems/libgnome-vfs.so. This in turn loads and links gnome-vfs itself, FAM, libgnomeui, libgnome, libbonobo, libORBit, etc. How the hell do we make this faster?
-
Append home, desktop. This is what puts your Home and your Desktop folders at the top of the shortcuts pane. It takes a lot more time than the rest of the volumes and bookmarks becaues this is where the machinery for icons and MIME-types gets loaded for the first time. Caveat: I'm pretty sure that my icon and MIME caches are not set up right. This is from a straight jhbuild, however — maybe we should make the packages update the caches when doing make install?
-
Append volumes. Fast; no worries about this.
-
Append bookmarks. Let's say the goal is to set up a file chooser in 0.1 seconds — then we can't be taking 0.13 seconds to set up just the bookmarks!
-
Set to cwd. If you don't call gtk_file_chooser_set_current_folder() after creating a file chooser, it will show the current directory by default. This is good, except that right now the chooser always loads the cwd at initialization time, even if you later set it to the right directory! This is bad for GNOME apps, since most of them get run from the panel menus, and their initial cwd is $HOME; most people's home directories are large, so setting them up is expensive. This does not really show up in this profile since my cwd was ~/src/file-chooser-tests, which only has a few files in it. I think I'll just add another state to the state machine which the file chooser uses for loading folders; that state will indicate that no folder has been set yet and we should load the cwd only until absolutely necessary.
-
Style stuff. There's a lot going on when the widget gets realized. It rescans its icons to make them appropriate for the theme, reloads the icons twice (!), and generally does a lot of in-memory work. I think we are calling shortcuts_reload_icons() from more than one place in the per-style and per-screen functions.
-
Set to right directory. Not bad at 0.03 seconds, but this is for an empty directory. There's a lot of stat()ing going on of the parent folders, as well as a bunch of FAM traffic.
-
Reload folder. The current folder gets re-read when the file chooser gets mapped. This is for apps like the GIMP which create a single file chooser and unmap/remap it between invocations, rather than creating it and destroying it every time. I just need a flag that says whether the folder is up to date to avoid reloading.
-
Reload bookmarks. This is done for the same reason. Again, it just needs a flag.
But the scary part is initialization. I haven't looked at that part of the strace log very closely, but it seems to be related almost exclusively to dynamic linking.
-
-
Bare-bones thumbnailer for Office files:
Documentation on how to install a thumbnailer, for the GNOME ISV guide.
The thumbnailer is in the libgsf module on CVS. It's very limited right now: it only supports MS Office files which were generated on Windows, and it will only generate thumbnails for files that have the VT_CF metadata property — neither OpenOffice nor Gnumeric save that data, so MS Office files saved with those will not work yet. Also, the thumbnailer assumes that convert(1) is in your $PATH, and is limited in ways like that. Even an unfriendly glance will make it fail right now.
-
In some excellent news, we'll be moving to a small but cute house pretty soon. We got the keys to it on Saturday! Yesterday Oralia and I went to start fixing things up: she put plaster over nail holes in the walls, I fixed light sockets; we cleaned up a bit and did some gardening. The tiny garden patch has some delectable herbs: basil, rosemary, epazote, acuyo... Yesterday Oralia made fish in acuyo sauce, which was great. It feels good to eat something with at least some ingredients that grew in your own garden.
-
I just added a new API to GtkFileChooser: a "do-overwrite-confirmation" property which you can use in save mode. With it, the file chooser will automatically present a confirmation dialog when the user may overwrite a file (bug #152850). This new feature is in CVS HEAD.
To enable it, you can simply call this:
GtkWidget *chooser; chooser = gtk_file_chooser_dialog_new ("Save file" parent, GTK_FILE_CHOOSER_ACTION_SAVE, GTK_STOCK_CANCEL, GTK_RESPONSE_CANCEL, GTK_STOCK_SAVE, GTK_RESPONSE_ACCEPT, NULL); gtk_file_chooser_set_do_overwrite_confirmation (GTK_FILE_CHOOSER (dialog), TRUE); if (gtk_dialog_run (GTK_DIALOG (chooser)) == GTK_RESPONSE_ACCEPT) save_the_file (gtk_file_chooser_get_filename (GTK_FILE_CHOOSER (chooser)));If you are using a language binding, you can simply set the "do-overwrite-confirmation" property to TRUE.
You must enable it if you want to use it, because the default for this property is FALSE. Why is that? It is to preserve compatibility with applications that already have their own custom confirmation dialogs — they don't need to change at all, and they'll keep working. Newer apps should take advantage of the new feature by turning on that property.
If all you need is a simple confirmation dialog, you can just turn on the property and be done with it; you need to read no further.
More sophisticated apps may want to extend the confirmation behavior. In addition to the boolean property mentioned above, the file chooser also has a new "confirm-overwrite" signal, which gets emitted when your custom confirmation hook has to run. For example, GEdit also prompts you if the file to which you are trying to save is read-only. The new API contemplates this case. You can do something like this:
static GtkFileChooserConfirmation confirm_overwrite_callback (GtkFileChooser *chooser, gpointer data) { char *uri; uri = gtk_file_chooser_get_uri (chooser); if (is_uri_read_only (uri)) { if (user_wants_to_replace_read_only_file (uri)) return GTK_FILE_CHOOSER_CONFIRMATION_ACCEPT_FILENAME; else return GTK_FILE_CHOOSER_CONFIRMATION_SELECT_AGAIN; } else return GTK_FILE_CHOOSER_CONFIRMATION_CONFIRM; /* fall back to the default dialog */ } ... chooser = gtk_file_chooser_dialog_new (...); gtk_file_chooser_set_do_overwrite_confirmation (GTK_FILE_CHOOSER (dialog), TRUE); g_signal_connect (chooser, "confirm-overwrite", G_CALLBACK (confirm_overwrite_callback), NULL); if (gtk_dialog_run (chooser) == GTK_RESPONSE_ACCEPT) ...; -
How did I make the funky "confirm-overwrite" signal handler return a custom enumeration? I used a GSignalAccumulator! It's the first time I've needed one, and it makes me happy to know why they are there.
-
Random thoughts on GConf:
-
"du -s ~/.gconf" gives 3 MB.
-
"gconftool-2 --dump /", then "pmap `pidof gconfd-2`" gives an RSS of 16 MB, with an rwx map of 14 MB. Mark says that it balloons because gconfd also reads the schemas.
-
"tar jcf ~/.gconf" gives me a 43 KB file. That's tiny.
-
"time tar cf - ~/.gconf > /dev/null" takes 0.3 seconds with a not-really-warm buffer cache.
Let's see. (1) and (3) together imply that the gconfd data in memory could conceivably be made very small — it isn't right now. What if it used a trie and shared GStringChunks to avoid duplicating the key names? Does it need to store any locale-specific description strings in memory, or could they be read from the schemas only if requested?
(4) means that we can read the entire ~/.gconf hierarchy really fast. So we could read through it once at startup, dump all the data into an memory-efficient data structure, and keep it around for the lifetime of gconfd. Gconfd would not have to re-read the individual XML files again.
This, of course, comes out of my ass with no further analysis of the current implementation :)
-
-
For the fine people who are working on Vino, VNC, and the like: Joel on his VNC thingy. That looks a lot like Monkeyhelp done the cool way.
-
Luis's post on swamps and superheroes led me to Telsa's account of the swamp talk, which in turn led me way back in time to the first Halloween memo from Microsoft. Ignoring the comments in green in the annotated memo — "we open source ninjas r00l; M$ is eeeeevil and doesn't get it" —, there are interesting things to note now that the memo is a few years old.
Let's start with the swamp. The swamps to be drained are thus:
-
General robustness problems. Fill up your diskspace whilst editing your preferences and GNOME as a whole does not handle the situation gracefully. Has anyone actually tested to see if we do better in this front? How much code doesn't check all the return values from system calls, or from gnome-vfs for file access? How many functions do a blind "/* FIXME */ return;" when they encounter an error, rather than propagating an error code or a nice GError upwards?
-
App-specific robustness problems. The swamp talk is from April 2002. That's GNOME 2.0 or 1.4, if I'm correct. The culprit apps are probably doing a lot better now. Does someone have any Bugzilla statistics?
-
Multimedia problems. It is 2005. JWZ has one more problem with Linux audio, gets pissed off for the last time, and buys a Mac.
-
Proxy configuration problems. [...] how many times should you have to enter and update all your proxy details? [...] So why do you have to update this for every app?
-
Documentation problems. When UNIX was first shipped, it came with documentation in the form of man pages for every program and every configuration file, all cross-referenced. [...] along came texinfo [...] Now we have DocBook as well! [...] things are less well cross-referenced than when they started, [...] nothing addressed that. Any solutions which try to address this will have to address all these formats, and retrofit things onto the old formats too. This is still valid.
-
Font problems. Mostly solved. You drag-and-drop a font into the undiscoverable fonts:///, and apps pick it up. But this only happened because a superhero got his hands dirty and did it, not because all projects suddenlly and automatically gelled into the same solution.
-
Mimetype hell. How many times has the MIME spec been rewritten? Why doesn't this work as well as it did with MC and GNOME 1.2?
-
Poor defaults. We have gotten definitely better in this respect.
-
Upstream, downstream. Upstream maintainers should not assume that the downstream vendors and shippers will fix stupid defaults. [...] Downstream shippers in their turn should tell the upstream maintainers about problems and fixes. We are doing a lot better, mainly because downstream shippers have become upstream maintainers in many cases (or vice-versa). Also, upstream has realized that vendors do have the correct fixes, even if they are sometimes a bit ugly, and it's worth the maintainer's time to polish that fix rather than re-do it from scratch. And the other way around: vendors have realized that it is a waste of time to maintain their own home-grown patches indefinitely, so they'd better get in the loop with what upstream is doing and do their work there. HAL + D-BUS +gnome-volume-manager is probably the nicest example of this: vendors have mostly gotten rid of their home-grown magicdev/suseplugger/mydistro-volume-thingy and just switched to using the new infrastructure from upstream. Mostly.
-
Customisation too specific. Xdefaults [...] GNOME preferences [...] KDE preferences [...] From the user's point of view, this is silly. We still have no common configuration architecture.
Now, jump back to the Halloween memo (search for "un-sexy work"). There is this gem (boldface is mine):
Un-sexy work
Another interesting thing to observe in the near future of OSS is how well the team is able to tackle the "unsexy" work necessary to bring a commercial grade product to life.
In the operating systems space, this includes small, essential functions such as power management, suspend/resume, management infrastructure, UI niceties, deep Unicode support, etc.
[...]
Integrative/Architectural work
Integrative work across modules is the biggest cost encountered by OSS teams. An email memo from Nathan Myrhvold on 5/98, points out that of all the aspects of software development, integration work is most subject to Brooks' laws.
Power management? We still suck. Suspend/resume? Likewise. Management infrastructure? Slowly appearing. UI niceties? We're getting there. Deep Unicode support? Done: another superhero.
Integrative work across modules: this is the big one. Freedesktop.org can provide a place for people to agree on things, but someone has to implement the standards to make apps conform to them. Superheroes are indeed scarce.
-
-
On a more positive note... my mom just retired, she is sick of Windows and viruses, and wanted to have Linux on her home machine. The Linux distro she took home from work was really old, and it left her machine in a half-configured state. X wasn't working, and XConfigurator (remember that?) wasn't being helpful. So I told her over the phone, "you'll have to edit a configuration file by hand. Do you know any text editors?"
She nonchalantly replied, "yeah, vi is fine. Just tell me where the X configuration file lives". It turns out that my mom worked for a long time as a support technician for big-ass database software on Solaris/HP-UX/AIX, and she is moderately comfortable with vi. Go, mom.
So I gave her a copy of Fedora Core 3 a few weeks ago. The installer died on the third CD — after re-burning the CD, it kept dying there. Then I waited a few days for FC4 to come out. The installer in the first CD is hosed; the kernel oopses twice, consistently, before reaching the fourth screen (once at initrd time, once during an IDE spinlock).
Eventually I mailed her a copy of Ubuntu, which she installed without problems. Today she phoned me: "how do I configure my modem?"
It's a winmodem. Big sigh.
-
Today is our fourth wedding anniversary. To start the day well and restore balance to the universe, we made divorced eggs — fried, over tortillas, where one egg goes with red salsa and the other one with green salsa. Add refried beans and some tortilla chips for the garnishing.
-
My office is in a small room on the roof of our house, next to the clotheslines and the washing paraphernalia. In that room I keep three desks for the computers, a small folding table, a bookcase, and other random odds and ends that invariably get stowed away there, rather than cluttering the rest of the house. So, the office can get a bit crowded. To add to the mess, I used to keep my synthesizer by the large window in the office room.
Playing the piano there was not pleasant at all. There's the constant hum of the computers, which I find tremendously distracting. There's also the noise from the street. The result is that I did not play as much as I wanted, which is annoying by itself — you know that nagging feeling that comes when you have let something slip by without you doing anything about it? So, on Saturday I finally got off my ass and moved the synth downstairs into the bedroom that we use as a study/library. It is very quiet, and there are no distractions. Playing the piano there is so much better. I think I've played more during the past two days than I did over the last four months.
Now I need a piano bench. The chairs we have are not tall enough, and lowering the synth's stand by one step makes it just a bit too low. But the main thing is that now I have a comfortable environment to play.