========== working ==========

$ autopilot3 run unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary -vvv
15:10:01.733 INFO run:240 - Autopilot Source Version: 1.5.0
Autopilot Package Version: 1.5.1+15.04.20150908-0ubuntu1
Loading tests from: /usr/lib/python3/dist-packages

Tests running...
15:10:02.064 INFO _logging:40 - ************************************************************
15:10:02.065 INFO _logging:41 - Starting test unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary (Aquaris E4.5 Ubuntu Edition)
15:10:02.075 DEBUG utilities:44 - Possible backends: X11,UInput,OSK
15:10:02.082 WARNING utilities:57 - Can't create backend X11: ImportError("No module named 'Xlib'",)
15:10:02.338 DEBUG utilities:44 - Possible backends: UInput
15:10:02.339 DEBUG utilities:44 - Possible backends: X11,UPA
15:10:02.343 WARNING utilities:57 - Can't create backend X11: ValueError('Namespace Gdk not available',)
15:10:02.475 DEBUG utilities:44 - Possible backends: BAMF
15:10:02.491 WARNING utilities:57 - Can't create backend BAMF: ImportError("No module named 'Xlib'",)
15:10:02.493 WARNING testcase:181 - Process manager backend unavailable, application snapshot support disabled.
15:10:02.547 INFO __init__:194 - Lib path is '/usr/lib/arm-linux-gnueabihf/unity8', binary path is '/usr/bin/unity8'
15:10:02.548 INFO __init__:248 - Setting up LightDM mock lib
15:10:02.600 INFO __init__:257 - New library path: /usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/libusermetrics:/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/IntegratedLightDM/liblightdm:/vendor/lib:/system/lib
15:10:02.626 INFO __init__:84 - New QML2 import path: /usr/lib/arm-linux-gnueabihf/unity8/qml/mocks:/usr/lib/arm-linux-gnueabihf/qt5/imports
15:10:02.641 INFO __init__:231 - Starting unity
15:10:02.660 INFO logging:45 - str: Set the value of an initctl environment variable. Arguments (1,). Keyword arguments: {'global_': True}.
15:10:02.699 INFO process_helpers:93 - Starting job unity8 with arguments ('BINARY=/usr/bin/unity8', 'XDG_DATA_DIRS=/usr/share/unity8/mocks/data:/usr/share/ubuntu-touch:/usr/local/share:/usr/share', 'QML2_IMPORT_PATH=/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks:/usr/lib/arm-linux-gnueabihf/qt5/imports', 'LD_LIBRARY_PATH=/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/libusermetrics:/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/IntegratedLightDM/liblightdm:/vendor/lib:/system/lib', 'ARGS=--mode=full-greeter', 'QT_LOAD_TESTABILITY=1').
15:10:03.759 INFO process_helpers:101 - unity8 start/running, process 31065

15:10:11.776 WARNING _object_registry:256 - Generating introspection instance for type 'b'unity8'' based on generic class.
15:10:11.782 DEBUG utilities:44 - Possible backends: UInput
15:10:11.783 DEBUG __init__:219 - Unity started, waiting for it to be ready.
15:10:11.847 DEBUG utilities:162 - 'GetState Query(b'/unity8//QQuickView')' took 0.063S
15:10:11.856 DEBUG utilities:44 - Possible backends: UInput
15:10:13.935 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView[id=2]//*[objectName="greeter"]')' took 2.072S
15:10:13.947 DEBUG utilities:44 - Possible backends: UInput
15:10:13.970 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickLoader/Greeter[id=3]')' took 0.020S
15:10:13.983 DEBUG __init__:221 - Unity loaded and ready.
15:10:15.091 DEBUG utilities:162 - 'GetState Query(b'/unity8//QQuickView')' took 0.046S
15:10:15.098 DEBUG utilities:44 - Possible backends: UInput
15:10:16.689 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView[id=2]//Notifications[objectName="notificationList"]')' took 1.590S
15:10:16.691 WARNING _object_registry:256 - Generating introspection instance for type 'b'Notifications'' based on generic class.
15:10:16.712 DEBUG utilities:44 - Possible backends: UInput
15:10:16.714 INFO __init__:74 - Creating ephemeral: summary(Upload of image completed), body(None), urgency('NORMAL') and Icon(/usr/share/unity8/graphics/applicationIcons/facebook.png)
15:10:16.865 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications[id=4]//Notification[objectName="notification1"]')' took 0.136S
15:10:16.867 WARNING _object_registry:256 - Generating introspection instance for type 'b'Notification'' based on generic class.
15:10:16.878 DEBUG utilities:44 - Possible backends: UInput
15:10:16.898 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.018S
15:10:16.930 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.019S
15:10:16.957 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.015S
15:10:16.983 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.015S
15:10:17.011 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.017S
15:10:17.045 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.021S
15:10:17.076 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.017S
15:10:17.106 DEBUG utilities:162 - 'GetState Query(b'/unity8/QQuickView/OrientedShell/Shell/QQuickItem/Notifications/QQuickItem/Notification[id=11]')' took 0.017S
15:10:17.119 INFO process_helpers:149 - Stoping job unity8.
15:10:19.250 INFO process_helpers:157 - unity8 stop/waiting

15:10:19.252 INFO logging:45 - str: Remove an initctl environment variable. Arguments (). Keyword arguments: {'global_': True}.
15:10:19.275 INFO testresult:44 - OK: unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary(Aquaris E4.5 Ubuntu Edition)

Ran 1 test in 17.268s
OK

========== install silo 22 ==========

# note you need citrain tool from overlay or install dmz-cursor-theme beforehand

$ citrain device-upgrade 22 0000

========== broken ==========

phablet@ubuntu-phablet:~$ autopilot3 run unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary -vvv
15:15:09.820 INFO run:240 - Autopilot Source Version: 1.5.0
Autopilot Package Version: 1.5.1+15.04.20150908-0ubuntu1
Loading tests from: /usr/lib/python3/dist-packages

Tests running...
15:15:10.189 INFO _logging:40 - ************************************************************
15:15:10.190 INFO _logging:41 - Starting test unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary (Aquaris E4.5 Ubuntu Edition)
15:15:10.194 DEBUG utilities:44 - Possible backends: X11,UInput,OSK
15:15:10.200 WARNING utilities:57 - Can't create backend X11: ImportError("No module named 'Xlib'",)
15:15:10.451 DEBUG utilities:44 - Possible backends: UInput
15:15:10.452 DEBUG utilities:44 - Possible backends: X11,UPA
15:15:10.457 WARNING utilities:57 - Can't create backend X11: ValueError('Namespace Gdk not available',)
15:15:10.670 DEBUG utilities:44 - Possible backends: BAMF
15:15:10.690 WARNING utilities:57 - Can't create backend BAMF: ImportError("No module named 'Xlib'",)
15:15:10.691 WARNING testcase:181 - Process manager backend unavailable, application snapshot support disabled.
15:15:10.738 INFO __init__:194 - Lib path is '/usr/lib/arm-linux-gnueabihf/unity8', binary path is '/usr/bin/unity8'
15:15:10.740 INFO __init__:248 - Setting up LightDM mock lib
15:15:10.790 INFO __init__:257 - New library path: /usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/libusermetrics:/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/IntegratedLightDM/liblightdm:/vendor/lib:/system/lib
15:15:10.817 INFO __init__:84 - New QML2 import path: /usr/lib/arm-linux-gnueabihf/unity8/qml/mocks:/usr/lib/arm-linux-gnueabihf/qt5/imports
15:15:10.832 INFO __init__:231 - Starting unity
15:15:10.851 INFO logging:45 - str: Set the value of an initctl environment variable. Arguments (1,). Keyword arguments: {'global_': True}.
15:15:10.889 INFO process_helpers:93 - Starting job unity8 with arguments ('BINARY=/usr/bin/unity8', 'ARGS=--mode=full-greeter', 'LD_LIBRARY_PATH=/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/libusermetrics:/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/IntegratedLightDM/l
iblightdm:/vendor/lib:/system/lib', 'QML2_IMPORT_PATH=/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks:/usr/lib/arm-linux-gnueabihf/qt5/imports', 'XDG_DATA_DIRS=/usr/share/unity8/mocks/data:/usr/share/ubuntu-touch:/usr/local/share:/usr/share', 'QT_LOAD_TESTABILITY=1').
15:15:12.085 INFO process_helpers:101 - unity8 start/running, process 5006

15:15:20.218 WARNING _object_registry:256 - Generating introspection instance for type 'b'unity8'' based on generic class.
15:15:20.224 DEBUG utilities:44 - Possible backends: UInput
15:15:20.225 DEBUG __init__:219 - Unity started, waiting for it to be ready.
15:15:20.284 DEBUG utilities:162 - 'GetState Query(b'/unity8//QQuickView')' took 0.057S
15:15:20.435 DEBUG utilities:44 - Possible backends: X11,UPA
15:15:20.437 WARNING utilities:57 - Can't create backend X11: ValueError('Namespace Gdk not available',)
15:15:21.625 INFO process_helpers:149 - Stoping job unity8.
15:15:34.123 INFO process_helpers:157 - unity8 stop/waiting

15:15:34.126 INFO logging:45 - str: Remove an initctl environment variable. Arguments (). Keyword arguments: {'global_': True}.
15:15:34.166 ERROR testresult:44 - ERROR: unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary(Aquaris E4.5 Ubuntu Edition)
15:15:34.168 ERROR testresult:44 - /var/log/syslog: {{{
Oct 19 15:15:10 ubuntu-phablet kernel: [  158.780488]input: py-evdev-uinput as /devices/virtual/input/input5
Oct 19 15:15:10 ubuntu-phablet powerd[986]: 158882 slow avg 270.648480 fast avg 271.570352 last 269.000000
Oct 19 15:15:10 ubuntu-phablet kernel: [  159.018429]input: autopilot-finger as /devices/virtual/input/input6
Oct 19 15:15:12 ubuntu-phablet powerd[986]: handle_requestWakeup from :1.128 - ubuntu push client, 1445260812
Oct 19 15:15:12 ubuntu-phablet powerd[986]: void {anonymous}::reset_alarm_clock() found 3 remaining wakeup requests
Oct 19 15:15:12 ubuntu-phablet powerd[986]: setting hardware wakeup time to 2015-10-19 15:17:59 for ubuntu push client
Oct 19 15:15:14 ubuntu-phablet powerd[986]: 162895 slow avg 269.029111 fast avg 268.510836 last 268.500000
Oct 19 15:15:14 ubuntu-phablet dbus[849]: [system] Activating service name='org.freedesktop.hostname1' (using servicehelper)
Oct 19 15:15:14 ubuntu-phablet dbus[849]: [system] Successfully activated service 'org.freedesktop.hostname1'
Oct 19 15:15:24 ubuntu-phablet powerd[986]: 172894 slow avg 269.197781 fast avg 268.949699 last 269.000000
}}}
15:15:34.170 ERROR testresult:44 - traceback: {{{
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/test_notifications.py", line 461, in test_icon_summary
    self.launch_unity()
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/__init__.py", line 220, in launch_unity
    self.wait_for_unity()
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/__init__.py", line 286, in wait_for_unity
    greeter = self.main_window.wait_select_single(objectName='greeter')
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/__init__.py", line 300, in main_window
    return self._proxy.select_single(shell.QQuickView)
  File "/usr/lib/python3/dist-packages/autopilot/introspection/dbus.py", line 244, in select_single
    raise StateNotFoundError(type_name_str, **kwargs)
autopilot.exceptions.StateNotFoundError: Object not found with name 'QQuickView'.

Tips on minimizing the occurrence of this failure are available here: https://developer.ubuntu.com/api/autopilot/python/1.5.0/faq-troubleshooting/
}}}
15:15:34.172 ERROR testresult:44 - Binary attachment: "FailedTestScreenshot" (image/png)
======================================================================
ERROR: unity8.shell.tests.test_notifications.EphemeralNotificationsTests.test_icon_summary(Aquaris E4.5 Ubuntu Edition)
----------------------------------------------------------------------
Binary content:
  FailedTestScreenshot (image/png)

/var/log/syslog: {{{
Oct 19 15:15:10 ubuntu-phablet kernel: [  158.780488]input: py-evdev-uinput as /devices/virtual/input/input5
Oct 19 15:15:10 ubuntu-phablet powerd[986]: 158882 slow avg 270.648480 fast avg 271.570352 last 269.000000
Oct 19 15:15:10 ubuntu-phablet kernel: [  159.018429]input: autopilot-finger as /devices/virtual/input/input6
Oct 19 15:15:12 ubuntu-phablet powerd[986]: handle_requestWakeup from :1.128 - ubuntu push client, 1445260812
Oct 19 15:15:12 ubuntu-phablet powerd[986]: void {anonymous}::reset_alarm_clock() found 3 remaining wakeup requests
Oct 19 15:15:12 ubuntu-phablet powerd[986]: setting hardware wakeup time to 2015-10-19 15:17:59 for ubuntu push client
Oct 19 15:15:14 ubuntu-phablet powerd[986]: 162895 slow avg 269.029111 fast avg 268.510836 last 268.500000
Oct 19 15:15:14 ubuntu-phablet dbus[849]: [system] Activating service name='org.freedesktop.hostname1' (using servicehelper)
Oct 19 15:15:14 ubuntu-phablet dbus[849]: [system] Successfully activated service 'org.freedesktop.hostname1'
Oct 19 15:15:24 ubuntu-phablet powerd[986]: 172894 slow avg 269.197781 fast avg 268.949699 last 269.000000
}}}

test-log: {{{
15:15:10.194 DEBUG utilities:44 - Possible backends: X11,UInput,OSK
15:15:10.200 WARNING utilities:57 - Can't create backend X11: ImportError("No module named 'Xlib'",)
15:15:10.451 DEBUG utilities:44 - Possible backends: UInput
15:15:10.452 DEBUG utilities:44 - Possible backends: X11,UPA
15:15:10.457 WARNING utilities:57 - Can't create backend X11: ValueError('Namespace Gdk not available',)
15:15:10.670 DEBUG utilities:44 - Possible backends: BAMF
15:15:10.690 WARNING utilities:57 - Can't create backend BAMF: ImportError("No module named 'Xlib'",)
15:15:10.691 WARNING testcase:181 - Process manager backend unavailable, application snapshot support disabled.
15:15:10.738 INFO __init__:194 - Lib path is '/usr/lib/arm-linux-gnueabihf/unity8', binary path is '/usr/bin/unity8'
15:15:10.740 INFO __init__:248 - Setting up LightDM mock lib
15:15:10.790 INFO __init__:257 - New library path: /usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/libusermetrics:/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/IntegratedLightDM/liblightdm:/vendor/lib:/system/lib
15:15:10.817 INFO __init__:84 - New QML2 import path: /usr/lib/arm-linux-gnueabihf/unity8/qml/mocks:/usr/lib/arm-linux-gnueabihf/qt5/imports
15:15:10.832 INFO __init__:231 - Starting unity
15:15:10.851 INFO logging:45 - str: Set the value of an initctl environment variable. Arguments (1,). Keyword arguments: {'global_': True}.
15:15:10.889 INFO process_helpers:93 - Starting job unity8 with arguments ('BINARY=/usr/bin/unity8', 'ARGS=--mode=full-greeter', 'LD_LIBRARY_PATH=/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/libusermetrics:/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks/IntegratedLightDM/l
iblightdm:/vendor/lib:/system/lib', 'QML2_IMPORT_PATH=/usr/lib/arm-linux-gnueabihf/unity8/qml/mocks:/usr/lib/arm-linux-gnueabihf/qt5/imports', 'XDG_DATA_DIRS=/usr/share/unity8/mocks/data:/usr/share/ubuntu-touch:/usr/local/share:/usr/share', 'QT_LOAD_TESTABILITY=1').
15:15:12.085 INFO process_helpers:101 - unity8 start/running, process 5006

15:15:20.218 WARNING _object_registry:256 - Generating introspection instance for type 'b'unity8'' based on generic class.
15:15:20.224 DEBUG utilities:44 - Possible backends: UInput
15:15:20.225 DEBUG __init__:219 - Unity started, waiting for it to be ready.
15:15:20.284 DEBUG utilities:162 - 'GetState Query(b'/unity8//QQuickView')' took 0.057S
15:15:20.435 DEBUG utilities:44 - Possible backends: X11,UPA
15:15:20.437 WARNING utilities:57 - Can't create backend X11: ValueError('Namespace Gdk not available',)
15:15:21.625 INFO process_helpers:149 - Stoping job unity8.
15:15:34.123 INFO process_helpers:157 - unity8 stop/waiting

15:15:34.126 INFO logging:45 - str: Remove an initctl environment variable. Arguments (). Keyword arguments: {'global_': True}.
}}}

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/test_notifications.py", line 461, in test_icon_summary
    self.launch_unity()
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/__init__.py", line 220, in launch_unity
    self.wait_for_unity()
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/__init__.py", line 286, in wait_for_unity
    greeter = self.main_window.wait_select_single(objectName='greeter')
  File "/usr/lib/python3/dist-packages/unity8/shell/tests/__init__.py", line 300, in main_window
    return self._proxy.select_single(shell.QQuickView)
  File "/usr/lib/python3/dist-packages/autopilot/introspection/dbus.py", line 244, in select_single
    raise StateNotFoundError(type_name_str, **kwargs)
autopilot.exceptions.StateNotFoundError: Object not found with name 'QQuickView'.

Tips on minimizing the occurrence of this failure are available here: https://developer.ubuntu.com/api/autopilot/python/1.5.0/faq-troubleshooting/

Ran 1 test in 24.005s
FAILED (failures=1)