module: don't use stop_machine on module load

Kay Sievers <kay.sievers@vrfy.org> discovered that boot times are slowed
by about half a second because all the stop_machine_create() calls,
and he only probes about 40 modules (I have 125 loaded on this laptop).

We only do stop_machine_create() so we can unlink the module if
something goes wrong, but it's overkill (and buggy anyway: if
stop_machine_create() fails we still call stop_machine_destroy()).

Since we are only protecting against kallsyms (esp. oops) walking the
list, synchronize_sched() is sufficient (synchronize_rcu() is probably
sufficient, but we're not in a hurry).

Kay says of this patch:
	... no module takes more than 40 millisecs to link now, most of
	them are between 3 and 8 millisecs.

	That looks very different to the numbers without this patch
	and the otherwise same setup, where we get heavy noise in the
	traces and many delays of up to 200 millisecs until linking,
	most of them taking 30+ millisecs.

Tested-by: Kay Sievers <kay.sievers@vrfy.org>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

diff --git a/kernel/module.c b/kernel/module.c
index e8cf636e614bafbd773397fcae32830291ae44bd..1a9a3986b136a47ae68d5255abaed6ed032d4bfe 100644 (file)
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1912,12 +1912,6 @@ static noinline struct module *load_module(void __user *umod,
        if (len > 64 * 1024 * 1024 || (hdr = vmalloc(len)) == NULL)
                return ERR_PTR(-ENOMEM);
 
-       /* Create stop_machine threads since the error path relies on
-        * a non-failing stop_machine call. */
-       err = stop_machine_create();
-       if (err)
-               goto free_hdr;
-
        if (copy_from_user(hdr, umod, len) != 0) {
                err = -EFAULT;
                goto free_hdr;
@@ -2303,12 +2297,13 @@ static noinline struct module *load_module(void __user *umod,
        /* Get rid of temporary copy */
        vfree(hdr);
 
-       stop_machine_destroy();
        /* Done! */
        return mod;
 
  unlink:
-       stop_machine(__unlink_module, mod, NULL);
+       /* Unlink carefully: kallsyms could be walking list. */
+       list_del_rcu(&mod->list);
+       synchronize_sched();
        module_arch_cleanup(mod);
  cleanup:
        kobject_del(&mod->mkobj.kobj);
@@ -2331,7 +2326,6 @@ static noinline struct module *load_module(void __user *umod,
        kfree(args);
  free_hdr:
        vfree(hdr);
-       stop_machine_destroy();
        return ERR_PTR(err);
 
  truncated: