Significant Performance Improvements with Qt 3D Studio 2.4

Speed of the 3D rendering is essential for a 3D engine, in addition to efficient use of system resources. The upcoming new Qt 3D Studio 2.4 release brings a significant boost to rendering performance, as well as provides further savings on CPU and RAM utilization. With our example high-end embedded 3D application the rendering speed is improved whopping 565%, while the RAM use and CPU load are down 20% and 51% respectively. 

Performance is a key driver for Qt and especially important for being able to run complex 3D applications on embedded devices. We have been constantly improving the resource efficiency with earlier releases of Qt 3D Studio and with the upcoming Qt 3D Studio 2.4 takes a major step forward in rendering performance. The exact performance increase depends a lot on the application and used hardware, so we have taken two example applications and embedded hardware for a closer look in this blog post. The example applications used in this post are automotive instrument clusters, but similar improvement can be seen in any application using Qt 3D Studio runtime.

Entry-level embedded example with Renesas R-Car D3

The entry-level embedded device used in the measurement is Renesas R-Car D3, which has the Imagination PowerVR GE8300 entry-class GPU (https://www.imgtec.com/powervr-gpu/ge8300/) and one ARM Cortex A53 CPU core. Operating system is Linux.

The example application used is the low-end cluster, available at https://git.qt.io/public-demos/qt3dstudio/tree/master/LowEndCluster. The low-end cluster example is well optimized, as described in a detailed blog post about optimizing 3D applications.

lowendcluster

 

In order to make the application as lightweight as possible, only the ADAS view is created as a real-time 3D user interface. Other parts of the instrument cluster are created with Qt Quick. This allows having a real-time 3D user interface even on a entry-class hardware like Renesas R-Car D3.

High-end embedded example with NVIDIA Tegra X2

The high-end embedded device used in the measurement is NVIDIA Jetson TX2 development board equipped with Tegra X2 SoC, which has 256-core NVIDIA Pascal™ GPU and Dual-Core NVIDIA Denver 2 64-Bit as well as Quad-Core ARM Cortex-A57 MPCore CPUs. Operating system is Linux.

The example application used is the Kria cluster, available at https://git.qt.io/public-demos/qt3dstudio/tree/master/kria-cluster-3d-demo. The Kria cluster example is made intentionally heavy with large and not fully optimized textures, high resolution etc.

kria3dclusterdemo

In the high-end example all the gauges and other elements are real-time 3D, rendered with the Qt 3D Studio runtime. There are very few Qt Quick parts and these are brought into the 3D user interface using texture sharing via QML streams.

Rendering performance improvement

The biggest improvement with the new Qt 3D Studio 2.4 release is to the rendering performance – getting the same application to render more Frames Per Second (FPS) on the same hardware. As always with Qt we aim to run steady 60 FPS, but on embedded devices pure performance is not enough. When there are items like heat management and tackling different usage scenarios it typically pays off not to run on the very edge of the SoC’s graphics capabilities. In the case of an application such as an instrument cluster, the performance needs to be smooth in all operation conditions, including under maximum load of the system. For measurement purposes with the high-end example we have disabled vsync, allowing the system to draw as many frames it can. In a typical real-life application there always is the vsync set, so anything that we can go over 60 FPS means saved processing resources.

The graphs below show the measured Frames Per Second with the high-end example on NVIDIA TX2 (vsync off) and with the low-end example on Renesas R-Car D3 (vsync on):

studio_fps

High end example: With the new Qt 3D Studio 2.4 we see a a whopping 565% improvement in the rendering performance. With Qt 3D Studio 2.3 the application was running only at 20 FPS, but the new Qt 3D Studio 2.4 allows the application to run 133 FPS. This is measured turning off vsync, just to measure the capability of the new runtime. In practice running 60 FPS is enough, and the additional capacity of the processor can be leveraged to have a larger screen (or another screen) or more complex application – or simply by not using the maximal capacity of the SoC to save on power.

Low-end example: The improvement is 46% because the maximum FPS is capped to 60 FPS by Qt Quick. With Qt 3D Studio 2.3 the application achieved 41 FPS, and with the new 2.4 runtime it reaches 60 FPS easily. Just like with the more powerful high-end hardware the excess capacity of the SoC can be used for running a more complex 3D user interface, or simply left unused.

CPU load improvement

The overall CPU load of an application is a sum of multiple things, one of them being the load caused by the 3D engine. In embedded applications it is important that using 3D in the application does not cause excessive load for the CPU. If the application exceeds the available CPU, it will not be able to render at target FPS and stuttering or other artefacts may appear on the screen.

The graphs below show the measured CPU load with the high-end example on NVIDIA TX2 and with the low-end example on Renesas R-Car D3:

studio_cpu

High-end example: With the new Qt 3D Studio 2.4 we see a hefty 51% improvement in the CPU load compared to Qt 3D Studio 2.3 while at the same time the FPS goes from 20 FPS to 133 FPS. The overall load with the Runtime 2.3 is 167% (of total 400%) and with the Runtime 2.4 the load drops to 81%. Note that the increased rendering speed has its effect on the CPU load as well. With the vsync on and FPS capped to 60 FPS, the CPU load is 74%.

Low-end example: We see only a modest 5% improvement in the CPU load, mainly due to the application being mostly Qt Quick. But this is with FPS going from 41 FPS up to 60 FPS at the same time. It should also be noted that the CPU of R-Car D3 is not very powerful, so the increased FPS of the overall application has its effect to the overall CPU load.

Memory usage improvement

For any graphics and especially 3D it is the assets that typically takes most of the RAM. There are ways to optimize, most notably avoiding unnecessary level of detail and leveraging texture compression. For the purposes of this blog post, we do not leverage any specific optimization methods. The measurements are done with exactly the same application, no other changes than using a different version of the Qt 3D Studio runtime.

The graphs below show the measured RAM use with the high-end example on NVIDIA TX2 and with the low-end example on Renesas R-Car D3:

studio_ram

High-end example: With the new Qt 3D Studio 2.4 we see a reduction of 48MB compared to Qt 3D Studio 2.3. This is 20% reduction to the overall RAM usage of the application.

Low-end Example: In the simpler example the reduction of RAM use is 9MB when using the new 2.4 runtime. Percentage-wise this is is a 15% reduction to the overall RAM usage of the application.

How was this achieved?

The improvements are really big especially on embedded, so one may wonder what was changed in the new version? What we did is to use the same runtime architecture as with Qt 3D Studio 1.x releases instead of running on top of Qt 3D. The core logic of the 3D engine is still the same as before, but it is running directly on top of OpenGL instead of using Qt 3D. This provides significantly improved performance especially on embedded devices, but also on more powerful desktop systems. By running Studio’s 3D engine directly on top of OpenGL we avoid overhead in rendering and simplify the architecture. The simpler architecture translates to less internal signalling, less objects in memory and reduced synchronization needs between multiple rendering threads. All this has allowed us to make further optimizations over the Qt 3D Studio 1.x – and of course to bring the new features developed in the Qt 3D Studio 2.x releases on top of the OpenGL based runtime.

The change in 3D runtime does not require any changes for most projects. Just change the import statement (import QtStudio3D.OpenGL 2.4 instead of import QtStudio3D 2.3) and then recompilation with new Qt 3D Studio 2.4 is enough. As API and the parts of the 3D engine relevant for the application are the same as earlier, all the same materials, shaders etc work just like before. In the rare cases where some changes are needed e.g. for some custom material, these are rather small.

Get Qt 3D Studio 2.4

If you have not yet tried out the Qt 3D Studio 2.4 pre-releases, you should take that for a spin. It is available with the online installer under the preview node. Currently we have the third Beta release out and soon provide the Release Candidate. Final release is targeted to be out before end of June. Qt 3D Studio is available under both the commercial and open-source licenses.

The post Significant Performance Improvements with Qt 3D Studio 2.4 appeared first on Qt Blog.

Qt 5.12.4 Released with support for OpenSSL 1.1.1

Qt 5.12.4, the fourth patch release of Qt 5.12 LTS, is released today. Qt 5.12.4 release provides a number of bug fixes, as well as performance and other improvements. As an important new item it provides binaries build with OpenSSL 1.1.1, including the new TLS 1.3 functionality.

Compared to Qt 5.12.3, the new Qt 5.12.4 provides around 250 bug fixes. For details of the most important changes, please check the Change files of Qt 5.12.4.

The update to OpenSSL 1.1.1 is important to note for users leveraging OpenSSL in their applications. We wanted to update now as the earlier version of OpenSSL runs out of support at the end of the year and some platforms, such as Android, need the new one even sooner. Unfortunately OpenSSL 1.1 is binary incompatible with 1.0, so users need to switch to the new one and repackage their applications. One important functionality enabled by OpenSSL 1.1 is TLS 1.3 bringing significant cryptography and speed improvements. As part of the change, some old and insecure crypto algorithms have been removed and support for some new crypto algorithms added. For the users not leveraging OpenSSL in their applications, no actions are needed. OpenSSL is not included in a Qt application, unless explicitly so defined by the developer.

Going forward, Qt 5.12 LTS will receive many more patch releases throughout the coming years and we recommend all active developed projects to migrate to Qt 5.12 LTS. Qt 5.9 LTS is currently in ‘Strict’ phase and receives only the selected important bug and security fixes, while Qt 5.12 LTS is currently receiving all the bug fixes. Qt 5.6 Support has ended in March 2019, so all active projects still using Qt 5.6 LTS should migrate to a later version of Qt.

Qt 5.12.4 is now available via the maintenance tool of the online installer. For new installations, please download latest online installer from Qt Account portal or from qt.io Download page. Offline packages are available for commercial users in the Qt Account portal and at the qt.io Download page for open-source users. You can also try out the Commercial evaluation option from the qt.io Download page.

The post Qt 5.12.4 Released with support for OpenSSL 1.1.1 appeared first on Qt Blog.

Qt Support – Aligning chart views underneath each other

One thing that comes up occasionally in support is that when you have multiple chart views that it would be good to align them up underneath each other so that the actual plotted size is the same for both charts. If you have similar sizes for the axes then this is easily achieved as they will take up the same width and it will align for you. However, if this is not the case as it so often is, then we can use the margins to force it to have the sizes that we want. So let’s say that our chart currently looks like:

Chart with unaligned axes

If you are using C++ for your chart code then this can be achieved with the aid of a single slot, that can be connected to the QChart::plotAreaChanged() signal. To start off we need to prevent recursion from happening as we will be changing margins and so on but we want to let the internals still do their thing so we don’t want to block signals. So to do that we will just have a static boolean which we will set to true to indicate that we are in the middle of our calculations:

    void updatePlotArea(const QRectF &area)
    {
        static bool fixing = false;
        if (fixing)
            return;
        fixing = true;

The next thing we need to do is work out what chart is the best one to be used for aligning the others against, this means picking the one that has the largest left value for the plot area (i.e. the one that has the widest axis). We use the currently changed one as a starting point for this and if any are in fact better we change the margins for those as it will ensure we have the most size available for the plot area at any given point for the largest axis.

    QChart *bestChart = (QChart *)sender();
         QRectF bestRect = area;
         foreach(QChart *chart, charts) {
            if (chart->plotArea().left() > bestRect.left()) {
                bestChart = chart;
                bestRect = chart->plotArea();
                chart->setMargins(QMargins(20, 0, 20, 0));
             }
         }

Then, with the exception of the one that ends up being the best one we adjust the margins to ensure that it matches the best one so that they are aligned correctly by setting the margins to be the existing margin plus the difference between the best chart’s plot area and this chart’s plot area. Finally we send any posted events to ensure it is updated right away for us.

    foreach(QChart *chart, charts) {
        if (bestChart != chart) {
            const int left = chart->margins().left() +
                (bestRect.left() - chart->plotArea().left());
            const int right = chart->margins().right() +
                (chart->plotArea().right() - bestRect.right());
            chart->setMargins(QMargins(left, 0, right, 0));
        }
    }
    QApplication::sendPostedEvents();
    fixing = false;

This will give us the two charts aligned to look like:

after

As for doing this in QML, we can do something similar in a function which is called via onPlotAreaChanged.

    property bool fixing: false
    property var chartViews: [chartview, chartview_b]
    function updatePlotArea(chart, area) {
        if (fixing)
            return
        fixing = true
        var tmpChart
        var bestRect = chart.plotArea
        var bestChart = chart
        for (var i = 0; i  Math.ceil(bestRect.left) ||
           (Math.ceil(tmpChart.plotArea.left) === 
            Math.ceil(bestRect.left) &&
            Math.floor(tmpChart.plotArea.right) < 
            Math.floor(bestRect.right))) {
                bestChart = tmpChart;
                bestRect = tmpChart.plotArea;
            }
        }
        bestRect.left = Math.ceil(bestRect.left)
        bestRect.right = Math.floor(bestRect.right)
        for (i = 0; i < chartViews.length; i++) {
            tmpChart = chartViews[i]
            if (tmpChart !== bestChart) {
                var newLeft = 20 + bestRect.left -
                      Math.floor(tmpChart.plotArea.left);
                var newRight = 20 +
                      Math.ceil(tmpChart.plotArea.right) -
                      bestRect.right;
                tmpChart.margins.left = newLeft
                tmpChart.margins.right = newRight
            }
        }
        fixing = false;
    }

The only difference being is that we account for the fact that the plot area is using real values and margins are still integer based so we do some extra accounting for that as a result.

In other news from the support desk, to reiterate the value of reporting bugs directly to the support team as a customer. As a customer indicated the issue reported in https://bugreports.qt.io/browse/QTBUG-74523 we were able to quickly find a solution and also get that integrated in time for the next release. It had been reported earlier this year, but was given a lower priority because Qt Quick Controls 1 is deprecated and therefore no one had scheduled time to investigate it. As always, when a bug is reported to us from a customer it will increase the priority and in cases like this we are able to solve it much quicker as a result as the support team can spend time on it too.

The post Qt Support – Aligning chart views underneath each other appeared first on Qt Blog.

Faster link time for Qt WebAssembly

I have built Qt and various apps using emscripten so many times over the last couple of years, it isn't even funny.

One detractor with building Qt applications for the web using Qt for WebAssembly, is the time it takes to build the client application. Especially during linking, it takes a huge amount of time to produce a binary for Qt WebAssembly.
This is caused by all the magic that the linker in emscripten is doing.

Luckily, llvm can now produce wasm binaries directly so emscripten does not have to go through any additional steps to output wasm. Emscripten and Qt can utilize this feature to reduce the length of time it takes to link and emit a .wasm binary.

YAY! \0/

I did some very non scientific measurements on build time. Using my development machine with make -j8:

Qt (not including configure) textedit
default configure Qt for WebAssembly user 20m3.706s user 2m46.678s
using -device-option WASM_OBJECT_FILES=1 user 23m30.230s user 0m29.435s

The result is that building Qt takes a tad longer, but building the client application is significantly faster. Which means development iterations will be much faster. In this instance, I used the textedit example in Qt. The build time went from 2 minutes and 46 seconds for a default non object files build, to 29 seconds! Zip! Zoom! Zang! Dang! that's fast (for wasm)

Unfortunately, this is not currently done by default by either Qt, emscripten or llvm. There is a way to do this which involves building llvm from upstream and telling emscripten to use that.

For the emscripten and binaryen version, I tested the current 1.38.32

From this bug report https://bugreports.qt.io/browse/QTBUG-72537
Morten has fleshed out a method for doing this.

------------------------------------------------------------

Getting started:

(Note: this is "how-I-did-it", not necessarily "how-it-should-be-done".)
1. Clone the following repositories:

2. Check out the version you want to use 
  • emscripten and binaryen have matching emsdk version tags, for example "1.38.23".
  • llvm has its own version numbers, and must be synced up manually. Currently:
    • emscripten <= 1.38.23 : llvm 8.0 (8.0.0-rc2 branch)
    • emscripten > 1.38.23 : llvm 9.0 (master branch)
      (em++/emcc will complain if you have the incorrect llvm version)

3. Build
  • emscripten is all python, no build needed.
  • binaryen: "cmake -GNinja && ninja"
  • llvm:
    cmake -GNinja
    ../path/to/llvmcheckout
    -DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi;lld"
    -DCMAKE_BUILD_TYPE=Release
    -DLLVM_TARGETS_TO_BUILD=WebAssembly
    -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly
    (one of the WebAssembly targets may be redundant)

4. Configure Environment
I use the following:
export EMSDK="/Users/msorvig/dev/emsdks/emscripten-1.38.23"
export PATH="$EMSDK:$PATH"
export LLVM="/Users/msorvig/dev/emsdks/llvm-8.0.0-build/bin"
export BINARYEN="/Users/msorvig/dev/emsdks/binaryen-1.38.23"
export EM_CONFIG="/Users/msorvig/dev/emsdks/.emscripten-vanillallvm-1.38.23"
export EM_CACHE="/Users/msorvig/dev/emsdks/.emscripten-vanillallvm-cache-1.38.23"
Where .emscripten-vanillallvm-1.38.23 is a copy of the .emscripten config file that emsdk generates.

------------------------------------------------------------

You will need to adjust the various paths to your system, of course. 
Then to configure and build Qt, add -device-option WASM_OBJECT_FILES=1 to the normal Qt for WebAssembly configure line.

The one I normally use is:

configure -xplatform wasm-emscripten -developer-build -nomake tests -nomake examples -opensource -confirm-license -verbose -compile-examples -no-warnings-are-errors -release  -device-option WASM_OBJECT_FILES=1


Works like a charm for the Qt 5.13 and the 5.13.0 branches of the Qt source code repo. I tried this with the 5.13.0 beta4 WebAssembly binaries, but got:

wasm-ld: warning: function signature mismatch:
so a complete rebuild is required.

There is a chapter regarding Qt for WebAssembly in the book, Hands-On Mobile and Embedded Development with Qt 5


Introducing QtCoAP

I am happy to introduce the new QtCoAP library! It is the client-side implementation of the Constrained Application Protocol (CoAP) for the Internet of Things. The library provides quick and easy way of using the CoAP protocol in your cross-platform Qt applications. Big thanks to our partners at Witekio for development and contribution of the main functionality! As it has been announced earlier, QtCoAP will be available in the new Qt 5.13 release as a part of our Qt for Automation offering, together with other IoT protocol implementations, such as MQTTKNX and OPC UA.

What is CoAP?

CoAP was designed as a lightweight machine-to-machine (M2M) communication protocol that can run on devices with scarce memory and computing resources. It is based on the concept of RESTful APIs and is very similar to HTTP. CoAP has a client-server architecture and uses GET, POST, PUT and DELETE requests for interaction with the data. But unlike HTTP, it uses the lightweight UDP for the transport instead of TCP. Additionally, it supports some interesting features like multicast requests, resource discovery and observation.

Thanks to the low overhead and simplicity, CoAP  has become one of the popular IoT protocols to be used on the embedded devices. It acts as a sort of HTTP for the embedded world.

Overview of Qt CoAP Implementation

QtCoAP supports the following functionality:

  • Resource observation
  • Resource discovery
  • Group communication (multicast)
  • Blockwise transfers
  • Security

The library is really simple to use. You just need to create an instance of QCoapClient and connect its signals:

QCoapClient client;
connect(&client, &QCoapClient::finished, this, &CoapHandler::onFinished);
connect(&client, &QCoapClient::error, this, &CoapHandler::onError);

Now you are ready to send requests and receive replies:

// GET requests
client.get(QUrl("coap://127.0.0.1:5683/resource"));
// or simply
client.get(QUrl("127.0.0.1/resource"));

// PUT/POST requests
QFile file("data.json");
// ...
client.post(QUrl("127.0.0.1/resource"), file.readAll());
client.put(QUrl("127.0.0.1/resource"), file.readAll());

// DELETE requests
client.deleteResource(QUrl("127.0.0.1/resource"));

Using the QCoapRequest class you can pass options and customize your requests. For example:

QCoapRequest request;
request.setUrl(QUrl("127.0.0.1"));
request.setPayload(file.readAll());
request.addOption(QCoapOption::UriPath, "resource");
client.put(request);

CoAP also provides a publish-subscribe mechanism achieved via “observe” requests:

QCoapReply *observeReply = client.observe(QUrl("127.0.0.1/temperature"));
connect(observeReply, &QCoapReply::notified, this, &CoapHandler::onNotified);

Now your application will get notified whenever the “/temperature” resource changes.

What makes CoAP even more interesting, is the ability to find and discover CoAP resources. You can discover resources on the given host:

QCoapResourceDiscoveryReply
*discoverReply = client.discover(QUrl("127.0.0.1"));

Or in the entire network:

QCoapResourceDiscoveryReply
*discoverReply = client.discover();

This will send a multicast discovery request to the IPv4 CoAP multicast group. You can also run the discovery for the IPv6 nodes:

discoverReply = client.discover(QtCoap::MulticastGroup::AllCoapNodesIPv6LinkLocal);
// or
discoverReply = client.discover(QtCoap::MulticastGroup::AllCoapNodesIPv6SiteLocal);
...
connect(discoverReply, &QCoapResourceDiscoveryReply::discovered, this, &CoapHandler::onDiscovered);

You will get several discovery replies from each CoAP device in your network. For example:

Host 1:

RES: 2.05 Content
;rt="temperature-c";if="sensor";obs,
;rt="firmware";sz=262144

Host 2:

RES: 2.05 Content
;rt="temperature-c";if="sensor";obs,

This will indicate, that in your network you have 2 devices running CoAP servers: one of them is connected to temperature and light sensors and the other has only a temperature sensor.

Security

Last, but not least is security. QtCoAP library supports the following security modes:

  • Authentication via pre-shared keys.
  • Using X.509 certificates.

For securing the CoAP connection, you need to pass one of these modes when creating the client and configure it accordingly. For example:

QCoapClient secureClient(QtCoap::SecurityMode::PreSharedKey);
QCoapSecurityConfiguration config;
config.setIdentity("Client_identity");
config.setPreSharedKey("secretPSK");
secureClient.setSecurityConfiguration(config);

Please give us your feedback if you find this post interesting!

The post Introducing QtCoAP appeared first on Qt Blog.

Building and testing on multiple platforms – introducing minicoin

When working on Qt, we need to write code that builds and runs on multiple platforms, with various compiler versions and platform SDKs, all the time. Building code, running tests, reproducing reported bugs, or testing packages is at best cumbersome and time consuming without easy access to the various machines locally. Keeping actual hardware around is an option that doesn’t scale particularly well. Maintaining a bunch of virtual machines is often a better option – but we still need to set those machines up, and find an efficient way to build and run our local code on them.

Building my local Qt 5 clone on different platforms to see if my latest local changes work (or at least compile) should be as simple as running “make”, perhaps with a few more options needed. Something like

qt5 $ minicoin run windows10 macos1014 ubuntu1804 build-qt

should bring up three machines, configure them using the same steps that we ask Qt developers to follow when they set up their local machines (or that we use in our CI system Coin – hence the name), and then run the build job for the code in the local directory.

This (and a few other things) is possible now with minicoin. We can define virtual machines in code that we can share with each other like any other piece of source code. Setting up a well-defined virtual machine within which we can build our code takes just a few minutes.

minicoin is a set of scripts and conventions on top of Vagrant, with the goal to make building and testing cross-platform code easy. It is now available under the MIT license at https://git.qt.io/vohilshe/minicoin.

A small detour through engineering of large-scale and distributed systems

While working with large-scale (thousands of hosts), distributed (globally) systems, one of my favourite, albeit somewhat gruesome, metaphors was that of “servers as cattle” vs “servers as pets”. Pet-servers are those we groom manually, we keep them alive, and we give them nice names by which to remember and call (ie ssh into) them. However, once you are dealing with hundreds of machines, manually managing their configuration is no longer an option. And once you have thousands of machines, something will break all the time, and you need to be able to provision new machines quickly, and automatically, without having to manually follow a list of complicated instructions.

When working with such systems, we use configuration management systems such as CFEngine, Chef, Puppet, or Ansible, to automate the provisioning and configuration of machines. When working in the cloud, the entire machine definition becomes “infrastructure as code”. With these tools, servers become cattle which – so the rather unvegetarian idea – is simply “taken behind the barn and shot” when it doesn’t behave like it should. We can simply bring a new machine, or an entire environment, up by running the code that defines it. We can use the same code to bring production, development, and testing environments up, and we can look at the code to see exactly what the differences between those environments are. The tooling in this space is fairly complex, but even so there is little focus on developers writing native code targeting multiple platforms.

For us as developers, the machine we write our code on is most likely a pet. Our primary workstation dying is the stuff for nightmares, and setting up a new machine will probably keep us busy for many days. But this amount of love and care is perhaps not required for those machines that we only need for checking whether our code builds and runs correctly. We don’t need our test machines to be around for a long time, and we want to know exactly how they are set up so that we can compare things. Applying the concepts from cloud computing and systems engineering to this problem lead me (back) to Vagrant, which is a popular tool to manage virtual machines locally and to share development environments.

Vagrant basics

Vagrant gives us all the mechanisms to define and manage virtual machines. It knows how to talk to a local hypervisor (such as VirtualBox or VMware) to manage the life-cycle of a machine, and how to apply machine-specific configurations. Vagrant is written in Ruby, and the way to define a virtual machine is to write a Vagrantfile, using Ruby code in a pseudo-declarative way:

Vagrant.configure("2") do |config|
    config.vm.box = "generic/ubuntu1804"
    config.vm.provision "shell",
        inline: "echo Hello, World!"
end

Running “vagrant up” in a directory with that Vagrantfile will launch a new machine based on Ubuntu 18.04 (downloading the machine image from the vagrantcloud first), and then run “echo Hello, World!” within that machine. Once the machine is up, you can ssh into it and mess it up; when done, just kill it with “vagrant destroy”, leaving no traces.

For provisioning, Vagrant can run scripts on the guest, execute configuration management tools to apply policies and run playbooks, upload files, build and run docker containers, etc. Other configurations, such as network, file sharing, or machine parameters such as RAM, can be defined as well, in a more or less hypervisor-independent format. A single Vagrantfile can define multiple machines, and each machine can be based on a different OS image.

However, Vagrant works on a fairly low level and each platform requires different provisioning steps, which makes it cumbersome and repetitive to do essentially the same thing in several different ways. Also, each guest OS has slightly different behaviours (for instance, where uploaded files end up, or where shared folders are located). Some OS’es don’t fully support all the capabilities (hello macOS), and of course running actual tasks is done different on each OS. Finally, Vagrant assumes that the current working directory is where the Vagrantfile lives, which is not practical for developing native code.

minicoin status

minicoin provides various abstractions that try to hide many of the various platform specific details, works around some of the guest OS limitations, and makes the definition of virtual machines fully declarative (using a YAML file; I’m by no means the first one with that idea, so shout-out to Scott Lowe). It defines a structure for providing standard provisioning steps (which I call “roles”) for configuring machines, and for jobs that can be executed on a machine. I hope the documentation gets you going, and I’d definitely like to hear your feedback. Implementing roles and jobs to support multiple platforms and distributions is sometimes just as complicated as writing cross-platform C++ code, but it’s still a bit less complex than hacking on Qt.

We can’t give access to our ready-made machine images for Windows and macOS, but there are some scripts in “basebox” that I collected while setting up the various base boxes, and I’m happy to share my experiences if you want to set up your own (it’s mostly about following the general Vagrant instructions about how to set up base boxes).

Of course, this is far from done. Building Qt and Qt applications with the various compilers and toolchains works quite well, and saves me a fair bit of time when touching platform specific code. However, working within the machines is still somewhat clunky, but it should become easier with more jobs defined. On the provisioning side, there is still a fair bit of work to be done before we can run our auto-tests reliably within a minicoin machine. I’ve experimented with different ways of setting up the build environments; from a simple shell script to install things, to “insert CD with installed software”, and using docker images (for example for setting up a box that builds a web-assembly, using Maurice’s excellent work with Using Docker to test WebAssembly).

Given the amount of discussions we have on the mailing list about “how to build things” (including documentation, where my journey into this rabbit hole started), perhaps this provides a mechanism for us to share our environments with each other. Ultimately, I’d like coin and minicoin to converge, at least for the definition of the environments – there are already “coin nodes” defined as boxes, but I’m not sure if this is the right approach. In the end, anyone that wants to work with or contribute to Qt should be able to build and run their code in a way that is fairly close to how the CI system does things.

The post Building and testing on multiple platforms – introducing minicoin appeared first on Qt Blog.

Qt Design Studio 1.2 released

We are happy to announce the release of Qt Design Studio 1.2!

Qt Design Studio is a UI design and development tool that enables designers and developers to rapidly prototype and develop complex UIs. Both designers and developers use Qt Design Studio and this makes collaboration between the two a lot simpler and more streamlined. To get an impression, you should watch this video.

The most notable addition to Qt Design Studio 1.2 is the Qt Bridge for Sketch. This allows you to seamlessly import your designs to Qt Design Studio.
The Qt Bridge for Sketch is delivered with Qt Design Studio as a plugin that you can install to Sketch. At the moment the feature set of the Qt Bridge for Sketch is very similar to the  Qt Bridge for Photoshop and we already support symbols.

We created a short video to give an impression of the Qt Bridge for Sketch.

For details about how to install and use the  Qt Bridge for Sketch check our online documentation.

Qt Design Studio 1.2 Community Edition

With Qt Design Studio 1.2 we also offer a free to download and use Community Edition. The Community Edition can be downloaded from here. The Qt Bridge for Sketch and the Qt Bridge for Photoshop are not part of the Community Edition.

Since there have been many questions about the source code of Qt Design Studio I want to emphasize that Qt Design Studio is basically a different configured Qt Creator and that most of the source code is already part of  Qt Creator.
The Qt Bridge for Sketch and the Qt Bridge for Photoshop are closed source, though. In the timeframe of Qt Creator 4.10, we will clean up the way how we exchange the icons, rename the executable and product name.

Qt Creator 4.9 already contains the graphical timeline editor for example. Users who are primarily C++ developers can continue to use Qt Creator for their QML development and to integrate Qt Design Studio projects.

Qt Design Studio 1.2 Community Edition

Qt Design Studio 1.2 Community Edition

Complex gradients

In Qt Design Studio 1.2 we now support the more complex gradients from Qt Quick Shapes for the Qt Design Studio Components. Those advanced gradients like spherical and conical gradients are very useful when creating gauges and meters especially since their properties can also be animated. Finally, you are not limited to linear vertical gradients anymore when using Qt Design Studio to design content.

qds11

For Qt Design Studio 1.2 we also fixed many bugs and issues.
To learn more about the improvements and fixes in Qt Design Studio 1.2, you can check out the changelog.

Download and try out Qt Design Studio 1.2

The commercially licensed packages are available through the online installer and on the Qt Account Portal. You can try out and evaluate Qt Design Studio by registering for an evaluation license. 2019).
The Community Edition, which does not contain the Qt Bridges, can be downloaded from here.

Getting Started

You can find the latest online documentation for Qt Design Studio 1.2 here. The documentation is also available from inside Qt Design Studio.

For Qt Design Studio we created step by step tutorials as part of the documentation.

The welcome page of Qt Design Studio contains examples and links to video tutorials to help you get started.

Please post issues you find or suggestions you have in our bug tracker.

The post Qt Design Studio 1.2 released appeared first on Qt Blog.

Why did Trolls and Snakes start to play together? (Qt at PyCon US)

After the successful release of Qt for Python, we have been aiming to bring the Python community to our amazing ecosystem, and at the beginning of May, we had the opportunity to attend the biggest Python Conference in the world, PyCon US.

Why Python?

Python and Qt were born around the same years, and one of the most important principles for both projects was that readability counts.

Since the adoption by the scientific community and the growing of its web framework communities like Django, Python has been getting more and more attention.
Nowadays, thanks to its principles and easy-to-use philosophy, Python has been playing an important role with the hot-topics we currently hear, like Data Analysis, Machine Learning, Artificial Intelligence, among others, which has been reflected in the latest Stack Overflow Surveys.
The results for this years survey positioned Python as the:

  • 4th most popular language,
  • 2nd most loved language and
  • 1st most wanted language.

This is a nice motivation to join forces with the Python community since we have been putting a lot of efforts since circa 2008, when the PySide project started.

The conference

pycon-main
PyCon started in the year 2003 in the US, and after a small conference with only 200 attendees, it has been spreading all over the world.
This year the number of attendees was roughly 3200, which were spread among the five parallel tracks, and open spaces events that happened during the three main days of the conference.

pycon-qtSince the open spaces were running in parallel during the whole conference, we decided to give it a try, and add one just for Qt. To be honest, we didn’t expect many people, and even the idea of “Let’s work and read emails for one hour at least…” came to our mind, but to our surprise, once we approached the room, there were already people inside. Initially, we thought that we were in the wrong room, but no, every single person inside was interested in hearing about the Qt project. Many people joined due to the curiosity of having the Qt project on a Python conference, and also because people had many questions.
The beginning of the discussion was focused on an overview of the last years of development, features from the latest Qt version, and how PySide has been evolving.
Many people were aware of the first versions of PySide, and they were really happy that it was at the end a stable released officially supported by Qt.
We actively discussed how to improve the Qt for Python project, and how to get the attention of more Python developers.

Fortunately, that was not all.

PySide2 PyCon Lightning Talk

The day before the conference was ending, we got the opportunity of present a Lightning Talk of only 3 minutes, so it needed to have a clear message to both get Python developers’ attention, and also encourage them to try Qt for Python after the conference.
Due to the time limitation, we didn’t want to show a complicated example, so we went for one of the simples QPushButton/QLabel examples you can imagine.
The reception was quite positive, and after the lightning talks session many people approached us to tell us that they used Qt in the past, they were interested in trying it out, or simply to ask what else you could do with Qt.

Since the interaction could have been subjective, we wanted to see if by any reason we have affected the downloads we daily get from PyPi, and this were the results:
PySide2 v5.12.3 downloads

We are aware this is just starting, and we expect that over time, more Python developers will start to use Qt for the countless applications Python has, and then helping both the Python and Qt community to grow.

But most importantly, Python is currently a first-class citizen in the Qt ecosystem, and this will continue with the help of you all, so finally we can have an ecosystem where Trolls and Snakes live happily together.

The post Why did Trolls and Snakes start to play together? (Qt at PyCon US) appeared first on Qt Blog.

Qt Creator 4.9.1 released

We are happy to announce the release of Qt Creator 4.9.1 !

In this release we fixed some regressions in the generic highlighter and in the SFTP deployment to remote linux devices, as well as many other issues. Have a look at our change log for details.

Get Qt Creator 4.9.1

The opensource version is available on the Qt download page under “Qt Creator”, and you find commercially licensed packages on the Qt Account Portal. Qt Creator 4.9.1 is also available as an update in the online installer. Please post issues in our bug tracker. You can also find us on IRC on #qt-creator on chat.freenode.net, and on the Qt Creator mailing list.

The post Qt Creator 4.9.1 released appeared first on Qt Blog.

KDAB brings its world-class software services to the BlackBerry QNX Distributor and Value-Added Integrator Program

Houston, Texas, May 24th, 2019 — KDAB, the Qt®, OpenGL®, and C++® experts, today announced that it has joined the BlackBerry QNX Distributor & Value-Added Integrator Program, a worldwide network of value-added integrators, distributors, and reseller experts trained on QNX technology. Under the agreement, KDAB will provide integration and customization services for a range of BlackBerry QNX embedded software solutions in the automotive, biotech, medical, industrial, entertainment, government, and consumer industries.

As the global leader in software consulting for architecture, design, and development of Qt®, C++®, and 3D/OpenGL® applications, KDAB has been a BlackBerry QNX partner for many years. For example, KDAB worked on adding Qt® support to the QNX CAR Platform for Infotainment, which included training the QNX CAR engineers, creating the initial Qt® integration, and implementing the product’s showcase UI. In addition, KDAB implemented Qt® for the QNX Neutrino® RTOS, the BlackBerry 10 OS and maintained the Qt® port on the QNX Neutrino® RTOS. KDAB also created Photon migration tools and services to help QNX customers migrate their GUI to Qt® when the Photon micro-GUI reached end-of-life.

Under the agreement, KDAB will support BlackBerry QNX products including the QNX Neutrino® RTOS, QNX OS for Safety, and QNX OS for Medical. This benefits KDAB customers by providing access to safe and secure software through a trusted supplier with a proven track record of delivering QNX-based solutions.

KDAB is a unique partner due to the depth and breadth of their products and services, as well as their extensive experience working with BlackBerry QNX software at all levels,” said Kaivan Karimi, SVP and Co-Head, BlackBerry Technology Solutions, BlackBerry. “If the Industrial IoT is to succeed, it’s going to need to be built on safety-certified, mission-critical embedded software and together with KDAB we offer an unbeatable combination of capabilities and technologies that today’s forward thinking companies are looking for in their mission to deliver best-in-class devices and applications.”

KDAB’s global customer base, which includes industry leaders such as Airbus, Ericsson, IBM, Nokia, Shell, and Siemens, will continue to benefit from the company’s ability to deliver technically superior and innovative cross-platform software but with the added benefit of a safety-critical and resilient RTOS that helps guard against system malfunctions, malware, and cybersecurity breaches.

The BlackBerry QNX Distributor & Value-Added Integrator Program will allow KDAB to augment BlackBerry QNX’s successful embedded legacy to design and develop secure, mission-critical QNX-based solutions – ultimately accelerating customer time to market.

We’re excited to support this full-featured and highly reliable microkernel RTOS portfolio for our customers,” said Matthias Kalle Dalheimer, KDAB’s President and CEO. “This will enable us to provide even more robust and better integrated safety-critical solutions at a rapid pace. We’re also pleased to extend the technical support we currently offer QNX customers.”

To learn about the BlackBerry QNX Distributor and Value-Added Integrator program, visit https://ca.blackberry.com/partners.

About KDAB

KDAB is the global leader in software consultancy for architecture, development, and design of Qt®, C++®, and OpenGL® application across embedded, mobile, and desktop platforms. An ISO certified company, they build effective processes and tools as well as software products based on experience, delivering hundreds of software projects on time with the highest quality. They build runtimes and develop infrastructure for applications and user-interfaces running in cars, medical devices, mobile phones, tablets, industrial machines, oil wells, desktops, and more. Some of the more frequent tasks they are challenged with include harnessing the power of native engineering on constrained hardware, transforming visions into working software, redesigning software architectures, and mentoring engineers on best practices.

KDAB is also the market leader for training in Qt®, Modern C++® and OpenGL®, with offices throughout Europe and the Americas.

www.kdab.com

Media Contacts

Tobias Morell (Europe), Ann Fernandes (US): press@kdab.com

 

The post KDAB brings its world-class software services to the BlackBerry QNX Distributor and Value-Added Integrator Program appeared first on KDAB.

Little Trouble in Big Data – Part 1

A few months ago, we received a phone call from a bioinformatics group at a European university. The problem they were having appeared very simple. They wanted to know how to usemmap() to be able to load a large data set into RAM at once. OK I thought, no problem, I can handle that one. Turns out this has grown into a complex and interesting exercise in profiling and threading.

The background is that they are performing Markov-Chain Monte Carlo simulations by sampling at random from data sets containing SNP (pronounced “snips”) genetic markers for a selection of people. It boils down to a large 2D matrix of floats where each column corresponds to an SNP and each row to a person. They provided some small and medium sized data sets for me to test with, but their full data set consists of 500,000 people with 38 million SNP genetic markers!

The analysis involves selecting a column (SNP) at random in the data set and then performing some computations on the data for all of the individuals and collecting some summary statistics. Do that for all of the columns in the data set, and then repeat for a large number of iterations. This allows you to approximate the underlying true distribution from the discreet data that has been collected.

That’s the 10,000 ft view of the problem, so what was actually involved? Well we undertook a bit of an adventure and learned some interesting stuff along the way, hence this blog series.

The stages we went through were:

  1. Preprocessing
  2. Loading the Data
  3. Fine-grained Threading
  4. Preprocessing Reprise
  5. Coarse Threading

In this blog, I’ll detail stages 1 and 2. The rest of the process will be revealed as the blog series unfolds, and I’ll include a final summary at the end.

1. Preprocessing

The first thing we noticed when looking at the code they already had is that there is quite some work being done when reading in the data for each column. They do some summary statistics on the column, then scale and bias all the data points in that column such that the mean is zero. Bearing in mind that each column will be processed many times, (typically 10k – 1 million), this is wasteful to repeat every time the column is used.

So, reusing some general advice from 3D graphics, we moved this work further up the pipeline to a preprocessing step. The SNP data is actually stored in a compressed form which takes the form of quantizing 4 SNP values into a few bytes which we then decompress when loading. So the preprocessing step does the decompression of SNP data, calculates the summary statistics, adjusts the data and then writes the floats out to disk in the form of a ppbed file (preprocessed bed where bed is a standard format used for this kind of data).

The upside is that we avoid all of this work on every iteration of the Monte Carlo simulation at runtime. The downside is that 1 float per SNP per person adds up to a hell of a lot of data for the larger data sets! In fact, for the full data set it’s just shy of 69 TB of floating point data! But to get things going, we were just worrying about smaller subsets. We will return to this later.

2. Loading the data

Even on moderately sized data sets, loading the entirety of the data set into physical RAM at once is a no-go as it will soon exhaust even the beefiest of machines. They have some 40 core, many-many-GB-of-RAM machine which was still being exhausted. This is where the original enquiry was aimed – how to use mmap(). Turns out it’s pretty easy as you’d expect. It’s just a case of setting the correct flags so that the kernel doesn’t actually take a copy of the data in the file. Namely, PROT_READ and MAP_SHARED:

void Data::mapPreprocessBedFile(const string &preprocessedBedFile)
{
    // Calculate the expected file sizes - cast to size_t so that we don't overflow the unsigned int's
    // that we would otherwise get as intermediate variables!
    const size_t ppBedSize = size_t(numInds) * size_t(numIncdSnps) * sizeof(float);
 
    // Open and mmap the preprocessed bed file
    ppBedFd = open(preprocessedBedFile.c_str(), O_RDONLY);
    if (ppBedFd == -1)
        throw("Error: Failed to open preprocessed bed file [" + preprocessedBedFile + "]");
 
    ppBedMap = reinterpret_cast<float *>(mmap(nullptr, ppBedSize, PROT_READ, MAP_SHARED, ppBedFd, 0));
    if (ppBedMap == MAP_FAILED)
        throw("Error: Failed to mmap preprocessed bed file");
 
    ...
}

When dealing with such large amounts of data, be careful of overflows in temporaries! We had a bug where ppBedSize was overflowing and later causing a segfault.

So, at this point we have a float *ppBed pointing at the start of the huge 2D matrix of floats. That’s all well and good but not very convenient for working with. The code base already made use of Eigen for vector and matrix operations so it would be nice if we could interface with the underlying data using that.

Turns out we can (otherwise I wouldn’t have mentioned it). Eigen provides VectorXf and MatrixXf types for vectors and matrices but these own the underlying data. Luckily Eigen also provides a wrapper around these in the form of Map. Given our pointer to the raw float data which is mmap()‘d, we can use the placement new operator to wrap it up for Eigen like so:

class Data
{
public:
    Data();
 
    // mmap related data
    int ppBedFd;
    float *ppBedMap;
    Map<MatrixXf> mappedZ;
}
 
 
void Data::mapPreprocessBedFile(const string &preprocessedBedFile)
{
    ...
 
    ppBedMap = reinterpret_cast<float *>(mmap(nullptr, ppBedSize, PROT_READ, MAP_SHARED, ppBedFd, 0));
    if (ppBedMap == MAP_FAILED)
        throw("Error: Failed to mmap preprocessed bed file");
 
    new (&mappedZ) Map<MatrixXf>(ppBedMap, numRows, numCols);
}

At this point we can now do operations on the mappedZ matrix and they will operate on the huge data file which will be paged in by the kernel as needed. We never need to write back to this data so we didn’t need the PROT_WRITE flag for mmap.

Yay! Original problem solved and we’ve saved a bunch of work at runtime by preprocessing. But there’s a catch! It’s still slow. See the next blog in the series for how we solved this.

The post Little Trouble in Big Data – Part 1 appeared first on KDAB.

KDAB at CppCon 2019

CppCon is the annual, week-long face-to-face gathering for the entire C++ community – the biggest C++ event in the world. This year, for the first time, CppCon takes place in the stunning Gaylord Rockies Hotel and Convention Center in Aurora, Colorado, very near Denver International Airport.

KDAB is a Bronze sponsor at CppCon, and will be offering 2-day pre- and post-conference training courses.

Sign up for:

We’ll keep you updated about this exciting event.

The post KDAB at CppCon 2019 appeared first on KDAB.

exec on Qt WebAssembly

When porting applications to Qt for WebAssembly, there are a few issues to keep in mind.

One big issue is that you do not have a full operating system underneath for you to utilize.
What you get is the same sandbox in which javascript lives. In some aspects, it is somewhat similar to a single-threaded microcontroller where there is just one thread in which to operate, limited filesystem access and limited resources.

One of the unusual and tricky bits about using Qt for WebAssembly, is that the exec loop is not typical. The Emscripten main loop is co-operative, after each event has a turn to run, control is then returned to the browser. Any blocking of the event loop will mean the web page in which your application runs will become unresponsive.

This is all fine and dandy for simple applications that use one main loop, but gets complicated when you try to exec a secondary loop. To stop the execution of an event loop, emscripten throws an exception, which leads to all kinds of fun. It also means it never returns to the same place that you expect it. So any modal dialog that uses exec() will not return values. Less than ideal.

Take for instance QColorDialog. Typical use is as such:

    QColorDialog dlg(parent);
dlg.exec();
QColor color = dlg.selectedColor();
Which is basically what QColorDialog::getColor does.

... and does not work with Qt WebAssembly, because exec() does not return to the same place as you expect! The call to selectedColor will never get called.

What you can do is use a non-modal dialog with the show() or in the case of QColorDialog open() function and use Qt's signal/slot API to retrieve the selected QColor.

 QColorDialog *dlg = new QColorDialog(this);

connect(
dlg, &QColorDialog::colorSelected,
[=](const QColor &selectedColor) {
qDebug() << Q_FUNC_INFO << selectedColor;
});
dlg->open();

You can read more about the Emscripten execution environment at
https://emscripten.org/docs/api_reference/emscripten.h.html#browser-execution-environment

You can also learn more about Qt for WebAssembly and other things in the book I wrote:
Hands on Mobile and Embedded Development with Qt 5

Qt Design Studio 1.2 Beta released

We are happy to announce the beta release of Qt Design Studio 1.2.

Qt Design Studio is a UI design and development tool that enables designers and developers to rapidly prototype and develop complex UIs. Both designers and developers use Qt Design Studio and this makes collaboration between the two a lot simpler and more streamlined. To get an impression, you should watch this video.

The main addition for Qt Design Studio 1.2 is the Sketch Bridge. It is now possible to export your scenes from Sketch and import them to Qt Design Studio 1.2.

qds11

Qt Design Studio 1.2 also finally supports powerful gradients for Shape-based items. This means you can add and animate spherical, gradients, conical gradients and linear gradients and their properties.

beta1_2

The commercially licensed packages are available through the online installer and on the Qt Account Portal. You can try out and evaluate Qt Design Studio by registering for an evaluation license.

With Qt Design Studio 1.2 there is also a free community version available, which lacks the Photoshop and Sketch bridges, though. The Beta of the Qt Design Studio 1.2 Community Edition can be found here.

Getting Started

You can find the latest online documentation for Qt Design Studio 1.2 here. The documentation is also available from inside Qt Design Studio.

For Qt Design Studio we created new tutorials as part of the documentation.

The welcome page of Qt Design Studio contains examples and links to video tutorials to help you get started.

Please post issues you find or suggestions you have in our bug tracker.

The post Qt Design Studio 1.2 Beta released appeared first on Qt Blog.

Qt Installer Framework 3.1.1 Released

We are happy to announce the release of Qt IFW 3.1.1. Major reason for releasing version 3.1.1 is that it fixes the bug QTIFW-1322. The bug was that after installing with 3.1.0 offline installer, the created maintenancetool size was huge. This occurred in Linux and in Windows. There are also other fixes, the full list of changes can be found from ChangeLog.

Precompiled binaries for the Installer Framework can be downloaded from Qt Online Installer, sources and binaries can be found also in Qt Download page (open source) or in your Qt Account.

The post Qt Installer Framework 3.1.1 Released appeared first on Qt Blog.

Release 3.2.0: Update to Qt 5.12.3 with ECMAScript 7, Subscriptions, Image Picker and Qt Creator 4.8.2

Felgo 3.2.0 adds support for Qt 5.12.3 and Qt Creator 4.8.2, which brings many features and improvements. The new long term supported Qt release includes lots of fixes, new modules and adds JavaScript support for ECMAScript 7. The updated Felgo IDE version 4.8.2 is more stable and adds compatibility with the latest toolchains and platforms.

This post summarizes all of the highlights and changes of the release:

 

Important Update Note: To deploy for Android, the required NDK version now changed to NDK r19c. See the migration hints of this update for more information.

Update to Qt 5.12 LTS – Long Term Supported Release

Qt 5.12.3 is the third patch version for Qt 5.12 LTS, which is a long-term-supported release. It incorporates more than 2000 bug fixes since the previous Qt LTS version, Qt 5.9.7.

On top of Qt 5.12.3, the new LTS version will receive many patch releases throughout the coming years. Additional Qt tools like the latest Qt 3D Studio version are based on Qt 5.12 LTS as well.

What took us so long?

You might wonder why you had to wait so long for Qt 5.12. The reason is pretty simple, there were several bugs in all the official Qt 5.12 releases up until 5.12.3. Those bugs, especially on mobile, prevented a stable release. We need to make sure your apps keep working on all platforms, with every update. This is why we spent more time than usual, to report and fix issues for this major update.

QML Engine Improvements and JavaScript Support for ECMAScript 7

QML performance and memory consumption improved a lot over the previous Qt versions. Especially Qt 5.11 brought some major improvements when using the Qt Quick Compiler to compile QML code ahead of time. Qt 5.12 improves this even more. The following chart shows memory consumption of a Qt Quick Controls example project for different Qt versions:

qt512_qml_engine_performance

Qt 5.9.7 required 18.5 MB RAM, while Qt 5.12 now uses 13.0 MB RAM (with pre-compiled QML). This means that memory usage with Qt 5.12 improved by 30% compared to Qt 5.9.

Qt focuses a lot on improving the QML engine performance. You can read more about Qt 5.12 performance improvements here.

With the new release, you can also take advantage of major improvements to the JavaScript engine that supports QML. Previously compatible with ECMAScript 5, it now fully supports ECMAScript 7. This allows the usage of modern JavaScript and simplifies the integration of Javascript libraries. ECMAScript modules are now also supported and can be loaded both from C++ as well as QML/JS.

Improved Qt Quick Controls 2 and TableView

You can now use the TableView as another type of Item View in Qt Quick. It’s a lot more performant than its previous Qt Quick Controls implementation and is filling the last major gaps in the available set of views.

It is similar to a ListView but with multiple columns. Unlike the previous view of QtQuick Controls 1, it does not do any styling. The TableView provides the optimal solution for instantiating, pooling and reusing delegates on demand as you scroll the table. A TableModel for the view can be created with a custom C++ QAbstractTableModel implementation.

Also, various controls in Qt Quick Controls 2 gained new methods or new functionalities.

Qt Quick Input Handlers, New Tech Previews and More

The Pointer Handlers of Qt 5.11 are now renamed to Input Handlers and are fully supported as a first-class feature in Qt Quick. The Input Handlers simplify the creation of complex touch interactions, that used to be difficult to do with MouseArea and TouchArea alone. They can detect events even in cases of deep nesting.

This also applies for the TapHandler, which can detect taps and touch gestures. Unlike MouseArea, it can handle events in multiple nested Items at the same time:

qt512_touch_inputhandler_nested
import QtQuick 2.12
import Felgo 3.0

App {
 NavigationStack {
   Page {
     title: "Nested Touch Example"

     // Outer Rectangle
     Rectangle {
       anchors.centerIn: parent
       id: outerRect
       width: dp(200)
       height: dp(200)
       color: tapHandler.pressed ? "lightyellow" : "lightgreen"

       TapHandler {
         id: tapHandler
       }

       // Inner Rectangle
       Rectangle {
         id: innerRect
         anchors.centerIn: parent
         width: dp(75)
         height: dp(75)
         color: tapHandler2.pressed ? "lightblue" : "orange"

         TapHandler {
           id: tapHandler2
         }
       }
     }

   }
 }
}

It is also easy to make an Item draggable with the DragHandler:

qt512_touch_draghandler
import QtQuick 2.12
import Felgo 3.0

App {
 NavigationStack {
   Page {
     title: "Drag Example"

     // Draggable Rectangle
     Rectangle {
       // initial position
       x: (parent.width - width) / 2
       y: (parent.height - height) / 2

       width: dp(200)
       height: dp(100)
       color: dragHandler.active ? "lightyellow" : "lightgreen"

       AppText {
         id: text
         text: "Drag Me!"
         anchors.centerIn: parent
       }

       DragHandler { id: dragHandler }
     }

   }
 }
}

HoverHandler was added as a new type of handler for detecting mouse hover. As touchscreens generally do not offer hover events, in practice it detects a hovering mouse or tablet stylus.

The update also adds a new tech preview: you can have a look at the DelegateChooser type. It allows to use different delegates in item views like AppListView. Depending on role values from the model or the index, you can choose between multiple delegates:

qt512_delegatechooser
import QtQuick 2.12
import Felgo 3.0

import Qt.labs.qmlmodels 1.0 // tech preview import

App {
 NavigationStack {
   Page {
     title: "Multiple Delegates"

     AppListView {
       anchors.fill: parent
       model: [
         {
           text: "Apple",
           detailText: "A delicious fruit with round shape",
           type: "fruit"
         },

         {
           text: "Beer",
           type: "drink"
         },

         {
           text: "Orange",
           detailText: "Another fruit with round shape",
           type: "fruit"
         },

         {
           text: "Wine",
           type: "drink"
         }
       ]

       delegate: DelegateChooser {
         role: "type"

         DelegateChoice {
           roleValue: "fruit"

           // Delegate for "fruit" type
           SimpleRow {
             onSelected: text = "Clicked"
           }
         }

         DelegateChoice {
           roleValue: "drink"

           // Delegate for "drink" type
           Rectangle {
             width: parent.width
             height: dp(50)

             color: Theme.tintColor

             AppText {
               x: dp(16)
               anchors.verticalCenter: parent.verticalCenter
               text: modelData.text
               color: "white"
             }
           }
         }
       }
     }

   }
 }
}

There are many more additions you get with Qt 5.12.3, for example:

Also note that the following modules are part of the Qt 5.12 release, but are deprecated and considered for removal in subsequent releases of Qt:

You can also have a look at the official Qt 5.12 blog post or see the full list of changes here.

Subscriptions for iOS and Android

The in-app purchase plugin now supports subscriptions for iOS and Android apps. Today, subscriptions are the best way to monetize your app, offering recurring revenue from your users. Also Apple and Google only take a 15% share on subscriptions after 1 year, compared to the 30% share for regular in-app purchases, so that is another plus!

If you want to use subscriptions or the built-in virtual economy system, we recommend using the Felgo in-app purchase plugin instead of the default Qt Purchasing module.

Note: If you use the Store plugin in your application on iOS, you will need to add additional files with this update. Refer to the integration guide to see which files and where to get them.

ImagePicker Control and Qt Creator Designer Improvements

The new ImagePicker control allows to choose multiple images from the device gallery in one go. It uses a custom QML GridView, which accesses photos from the device gallery or camera roll.

imagepicker-control
import Felgo 3.0
import QtQuick 2.0

App {
 NavigationStack {
   Page {
     id: page
     title: qsTr("Choose Photos")

     // right bar item to accept selection
     rightBarItem: TextButtonBarItem {
       text: "Next"
       enabled: imagePicker.selectedCount > 0

       // your app will probably pop the photo selection page from the stack to move on
       // this example only logs the selected photos
       onClicked: console.debug("SELECTED:", JSON.stringify(imagePicker.selection))
     }

     // image picker view for photo selection
     ImagePicker {
       id: imagePicker
       anchors.fill: parent
     }
   }
 }
}

You can also build your own UI for picking photos. Call NativeUtils::fetchGalleryPhotos() to load a list of all device photos, which your view can then access with the NativeUtils::galleryPhotos property. To further work with individual images, use NativeUtils::getCachedAssetPath().

For Qt Creator design mode, you can now see better item names for all Felgo components:

update-320-qtc-designer

New Felgo IDE – Qt Creator 4.8.2

Along with the Qt 5.12.3 update, Felgo 3.2.0 also presents you with Qt Creator 4.8.2. The new IDE version is more stable and adds support for the latest build tools and platforms.

Improved Android Compatibility and Better iOS Device Support

On iOS, the IDE now better detects later iPhone models like the iPhone XS. The update also includes many improvements and fixes for macOS Mojave.

For Android, you can now connect and use the debugger on API level 24 and later. The new Qt Creator is also fully compatible with Android SDK 28.

Note: The Android Build Kit now uses the Clang compiler instead of GCC, which got removed with NDK r18b. Thus, older NDK versions are no longer supported and we recommend to upgrade to NDK r19c. You can download it here.

Make sure to configure the new NDK in the Qt Creator settings for Android:

qt512_android_ndk19

Before you can build projects, we also recommend to clean previous build folders and project settings:

  • Close Qt Creator
  • Delete the *.pro.user file of your project
  • Remove the previous project build folder
  • Open your project in Qt Creator and choose your Build Kits
  • Rebuild your project for Android

Qt 5.12 and Qt Creator 4.8 also add basic support for the Android ARMv8 architecture. However, there are no pre-built libraries for this architecture with Felgo 3.2.0 yet. If you require early access to Felgo Android ARMv8 support, don’t hesitate to get in touch.

Multiple Debuggers, Programming Language Support and More

Qt Creator 4.8.2 supports simultaneously running multiple debuggers. The debugger tool bar in the Debug mode has an additional pop up menu where you can switch between running debugger instances and the “preset” view which allows to start new debuggers.

On Windows, Qt Creator does no longer force the use of ANGLE for OpenGL on user applications. You can also expect less issues with antivirus programs on Windows systems.

There’s also a new experimental feature to support the Language Server Protocol (LSP), which adds basic support for many different programming languages to Qt Creator:

qtcreator48_lsp_client

You can read more about the improvements of Qt Creator 4.8 in the official release post.

More Features, Improvements and Fixes

Felgo 3.2.0 includes many more features, for example:

For all relevant changes, features and fixes of recent Felgo updates, please check out the change log.

How to Update Felgo

Test out these new features by following these steps:

  • Open the Felgo SDK Maintenance Tool in your Felgo SDK directory.
  • Choose “Update components” and finish the update process to get this release as described in the Felgo Update Guide.

Update Felgo

If you haven’t installed Felgo yet, you can do so now with the latest installer from here. Now you can explore all of the new features included in this release!

For a full list of improvements and fixes to Felgo in this update, please check out the change log!

 

 

 

More Posts Like This

Release 3.1.0: New Felgo Plugins Version, Unified App Configuration and FlickablePage

V-Play is Now Felgo – New Release & Roadmap

The post Release 3.2.0: Update to Qt 5.12.3 with ECMAScript 7, Subscriptions, Image Picker and Qt Creator 4.8.2 appeared first on Felgo.

Qt on CMake Workshop Summary – May ’19

This is a follow-up post to Qt on CMake Workshop Summary – Feb ’19

Intro

From May 2nd to May 3rd another Qt on CMake workshop was hosted at the KDAB premises in Berlin, where interested stakeholders from both The Qt Company and KDAB gathered together to drive the CMake build system in Qt further. Many of KDAB’s customers are using CMake in their Qt projects, so we are keen to see the CMake support for Qt improve and happy to help out to make it happen. The workshop was public, for anyone interested, but we had no external visitors this time. We’d be happy to have some more CMake enthusiasts or interested people in these workshops, so be sure to sign up for the next CMake workshop (watch the qt-development mailing list for this)!

This workshop in May was mostly intended to reassess what has happened in the wip/cmake branch of qtbase since the last workshop and to discuss any further work. We spent almost half of the first day just deciding how to approach certain things such as how the CMake build system port will affect the upcoming Qt6 work, which is currently gaining momentum as well. We had between 8 and 10 people present across the 2 day workshop, from KDAB and (mostly) The Qt Company.

Workshop summary

Excerpt of the top-level CMakeLists.txt in qtbase.git

First of all: Thanks to Alexandru Croitor for driving the porting efforts and for organizing sprints and meetings where interested people can keep track of the progress!

The workshop summary notes are also courtesy of Alexandru, let me try to quickly recap the most interesting bits:

CMake config files in Qt6 and beyond

One of the key considerations of the CMake config files installed as part of the upcoming Qt6 was that there should a) be the possibility to just be able to use CMake targets like Qt::Core (compared to Qt5::Core) and functions/macro names like qt_do_x() (instead of qt5_do_x()) to allow most applications to just pull in a Qt version of their choice and then use “versionless” CMake identifiers. This allows to upgrade Qt versions more easily, without a lot of search-replace in CMake code. Note that you can continue to use the version-specific identifiers as before. This is an additional feature.

But on the other hand we’d also like to keep the possibility to mix Qt version X and Qt version Y in the same CMake project. Think about a project where two executables are being built, one depending on Qt6, the other one a potential Qt7 version. This is not as cumbersome as you’d think; we experience a lot of customer projects where people have this setup. It might as well be the case during a porting project, where old code might still continue to use an older Qt version.

Consider this example (which is not fully implemented yet, but you get the idea):

### Usecase: application wants to mix both Qt5 and Qt6, to allow gradual porting 

set(QT_CREATE_VERSIONLESS_TARGETS OFF) 
find_package(Qt5 COMPONENTS Core Gui Widgets) # Creates only Qt5::Core 
find_package(Qt6 COMPONENTS Core Gui Widgets) # Creates only Qt6::Core 
target_link_libraries(myapp1 Qt5::Core)
target_link_libraries(myapp2 Qt6::Core)

### Usecase: application doesn't mix Qt5 and Qt6, but allows to fully switch to link against either Qt5 or Qt6 

set(MY_APP_QT_MAJOR_VERSION 6) # <- potentially set at command line by application developer
# set(QT_CREATE_VERSIONLESS_TARGETS ON) <- Default, doesn't need to be set 
find_package(Qt${MY_APP_QT_MAJOR_VERSION} COMPONENTS Core Gui Widgets) # Creates Qt5::Core and Qt::Core OR Qt6::Core and Qt::Core, based on the variable
target_link_libraries(myapp Qt::Core) # Just links to whatever Qt major version was requested

More details (and development notes from the past meetings):

After a lot of back and forth we actually found a straight-forward way to at least create the two namespaces in the CMake config files easily, see e.g.: https://codereview.qt-project.org/#/c/253169/

QMake will still be around in Qt6

As it stands, existing users of Qt and specifically users of QMake do not have to fear the CMake port too much, for now. The current bias is towards keeping the qmake executable (and the associated mkspec functionality) around for the Qt6 lifetime, as we’d obviously create a lot of additional porting effort for our users. During the Qt6 lifetime it would probably be wise to consider moving your pet project to a CMake build system, but only time will tell.

QMake is currently built via the CMake build system in the wip/cmake branch and already available for use. Upside-down world, right. Additionally, we’re looking into generating the QMake module .pri files using CMake as well. All this is definitely no witch craft but just needs dedicated people to implement all of it.

Further notes

You can find a lot more details on the Wiki, in case you are curious, I would not like to duplicate even more of the really comprehensive work log produced here: https://wiki.qt.io/CMake_Port/Development_Notes

If you would like to learn more about CMake, we are offering a one-day Introduction to CMake training at the KDAB training day as part of Qt World Summit in Berlin this year.

If you have comments or if you want to help out, please ideally post feedback on the Qt Project infrastructure. Send a mail to the qt-development mailing list or comment on the wiki page dedicated for the CMake port. Or just join us in the IRC channel #qt-cmake on Freenode!

 

The post Qt on CMake Workshop Summary – May ’19 appeared first on KDAB.

Qt Quick Performance Improvements with Qt 5.12 LTS (Updated for Qt 5.12.3)

We are continuously working on improving the performance and optimizing the memory consumption of Qt. One focus area for Qt 5.12 has been to reduce the memory consumption of the QML engine and tuning the JavaScript performance. Qt 5.12.3 provides further improvements especially for QML engine performance. 

This is an update for the earlier Qt 5.12 performance blog post from last November. Measurements have been done with Qt 5.12.3 release, which contains further improvements to Qt Quick and QML performance compared to the Qt 5.12 beta used in the previous post.

Qt 5.9 LTS already shows a great improvement of the overall performance compared to the previous long-term supported Qt 5.6 LTS release. These are summarized in a blog post about Performance Improvements with Qt 5.9 LTS and Qt Quick Performance Improvements on 64-bit ARM. With Qt 5.12 LTS we have continued to tune these further and taken a deeper look into the areas of QML engine memory consumption and JavaScript performance. In addition to Qt 5.6 LTS no longer supported, the performance of the Qt 5.12 LTS is a solid reason to upgrade.

QML Memory usage

Applications use memory for multiple things and to optimize the overall memory usage a good start is typically to look into the graphics assets, leveraging asset conditioning, the Qt Lite feature system to remove unused functionality and other techniques to optimize the memory usage. As it is very difficult for an application developer to affect the memory used by the QML engine itself, we have taken a deep look into how it can be reduced.

One important optimization done with Qt 5.12 LTS are to the data structures used in conjunction with compiled QML. The QML engine creates cache files that contain a dump of internal data structures on the fly when loading a .qml or .js file at run-time (named .qmlc and .jsc). If found, these files are mapped into the application process and used directly without requiring a compilation step. We have an option to create those cache files ahead of time while compiling the application. In that case, we don’t have quite the same degree of visibility into the type system as the QML engine has at run-time. This leads to a staged approach where we can’t resolve quite as much as when generating the data structures at run time, requiring us to do some additional work on top of the pre-generated cache files.

In Qt 5.12 LTS two changes are done to reduce the memory consumption when using such ahead-of-time generated cache files:

  • Avoid the re-creation of the indexed string table for the in-memory generated compilation unit and instead fall back to the memory mapped string table provided from the cache file.
  • Reduce the size of data structures so that the size of the files as well as their memory consumption is reduced.

In addition to the changes described above, we have done also multiple smaller improvements to reduce the memory consumption of the QML engine, while still keeping the performance on a good level (and actually improving quite a bit in some areas).

Actual memory usage of a Qt Quick applications depends a lot upon the application itself. In the comparison chart below, we have measured the QML engine related memory consumption for two different applications. We have used the Qt Quick Controls example to represent a Qt Quick application.

qml_engine_ram_use

For the example application we can see that Qt 5.6.3 uses 36.3 MB, Qt 5.9.7 uses 18.5 MB and Qt 5.12.3 uses 13.0 MB RAM (using pre-compiled QML). This means that memory usage with Qt 5.12 is 30% lower than Qt 5.9 and 64% lower than Qt 5.6 with the example application. These savings are available for all users of Qt. For commercial users Qt 5.6 and 5.9 offered a separate tool called Qt Quick Compiler that can be used for pre-compiling QML. Using the Qt Quick Compiler, the memory usage of Qt 5.6 drops to 14.4 MB (60% lower RAM usage than without). So even when using the Qt Quick Compiler with 5.6, upgrading to Qt 5.12 will lead to a very nice 10% improvement in memory usage.

As all applications are different it is best to check how big the benefit is for your application. We compared the QML engine memory use of Qt 5.6 and 5.9 using Qt Quick Compiler and Qt 5.12 using the pre-generated QML with a rather complex real-life Qt Quick application. In this measurement Qt 5.9 used 1.1 MB less RAM than Qt 5.6 (both with Qt Quick Compiler), and Qt 5.12 used a whopping 9.8 MB less RAM than Qt 5.6 (with Qt Quick Compiler). Especially for embedded systems with constrained resources or ones that run multiple applications this type of RAM savings in the QML engine, with no changes to the application code, are very beneficial.

JavaScript Performance

A major difference of Qt 5.12 LTS compared to Qt 5.9 LTS and Qt 5.6 LTS is the completely new compiler architecture of the QML engine introduced with Qt 5.11. The new pipeline has also enabled making some of the memory use optimizations described earlier in this post. With the new pipeline, we get a significant improvement in JavaScript performance, even with the addition of ECMAScript 7 support in Qt 5.12.

To compare the JavaScript performance for different Qt versions, we have run the same JavaScript benchmark with Qt 5.6.3, Qt 5.9.7 and Qt 5.12.3. The benchmark is run on 64-bit ARM processors using Linux operating system.

qt_5_12_3_js_comparison

To highlight the improvement with Qt 5.9 and 5.12 we have normalized the result of Qt 5.6 to 100.

Overall improvement (TotalScore) of Qt 5.9.7 compared to Qt 5.6.3 is 1785% and Qt 5.12.3 compared to Qt 5.6.3 the improvement is 2138%. Compared to Qt 5.6.3, the biggest performance improvement is seen due to supporting JIT on 64-bit ARMv8 architecture used in many modern embedded processors. With Qt 5.6 JIT is not supported on 64-bit ARM.

Qt 5.9 and Qt 5.12 both support JIT, so between these the improvements are mainly due to the new compiler architecture. Compared to Qt 5.9.7 the overall JavaScript performance improvement of Qt 5.12.3 is 19%.

Overall Qt Quick Performance

Because it is so easy to break performance, we regularly run several different kinds of performance tests. These also help in analyzing how the various optimizations work across different platforms and to make sure feature development and error corrections do not have unintended side effects to performance. Some of these results are visible at testsresults.qt.io, for example the QML Bench tests.

To compare the effect of Qt versions, we have run the full QML Bench test set with Qt 5.6.3, Qt 5.9.7 and Qt 5.12.3. To improve the comparability of results, we are running Qt Quick Controls 1 tests on Qt 5.6.3, as it does not have Qt Quick Controls 2 functionality. Otherwise the functionality tested by the QML Bench is same between these Qt versions. For these tests, QML Bench was run on an Nvidia Jetson TX1 development board (64-bit ARM processor) running the Nvidia supported Linux for Tegra Ubuntu 16.04.

Due to the very large performance improvement in Javascript and Qt Quick Controls, we have presented those separately to make the charts below more readable (JavaScript earlier in this post and Qt Quick Controls in an earlier blog). We have again normalized the Qt 5.6.3 result to be 100 in order to better highlight the improvements with Qt 5.9.7 and Qt 5.12.3.

qt_5_12_3_qml_comparison

For the overall QML Bench test the performance with Qt 5.9.7 is over double compared Qt 5.6.3 (109% better than Qt 5.6.3), and the performance of Qt 5.12.3 is even slightly higher (113% better than Qt 5.6.3). The improvements in areas other than JavaScript and Qt Quick Controls are smaller, typically between 15 to 25%. Comparing Qt 5.9.7 and Qt 5.12.3 shows overall improvement of 7%. This has improved since the first Qt 5.12 releases as the performance regressions in a few tests have been fixed. Those using Qt 5.12 LTS, should update to Qt 5.12.3 or later in order to get this increase in Qt Quick performance.

QML Bench has been developed with typical application use cases in mind, trying to cover as many of those as possible in a comprehensive suite. But as with any benchmark, this still means that the improvements in your application might vary depending on how QML is being used there – so it is always recommended to gather performance data with your own application to see how much its performance improves.

Other improvements

In this post the focus was in the Qt Quick performance, where we have improved the performance of Qt 5.12 LTS significantly compared to the earlier LTS releases of Qt in three important areas:

  • QML memory usage
  • Overall Qt Quick performance
  • JavaScript performance

In addition to these, Qt 5.12 LTS provides many other performance improvements as well. One important improvement in Qt 5.12 LTS is the possibility to use pre-generated distance fields of fonts. Earlier Qt has created the distance fields during the application startup, which can consume a lot of CPU cycles especially for non-latin fonts (that have many glyphs) and complex latin fonts (that have a lot of different shapes). With Qt 5.12 LTS, we release a tool called “Qt Distance Field Generator”, which allows you to pregenerate the distance fields for either a selection of the glyphs in a font or all of them.

Other areas where we have been improving performance include, for example, Qt 3D CPU usage and Qt 3D memory consumption (especially beneficial for Qt 3D Studio applications). Qt 5.12 LTS also introduces a completely re-designed TableView, which takes the performance of large tables to completely different level than the old TableView implementation of Qt Quick Controls 1. Of course, Qt 5.12 LTS also benefits from the compressed texture support introduced with Qt 5.10 and improved with Qt 5.11, as well as multiple other improvements done since the previous LTS release of Qt.

If you have not yet looked into Qt 5.12 LTS, please take it for a spin.

The post Qt Quick Performance Improvements with Qt 5.12 LTS (Updated for Qt 5.12.3) appeared first on Qt Blog.

Mobile and Embedded Development with Qt

At times it was a bit painful juggling writing a book, doing my day job and running around doing the things that life throws. It's done and dusted now, you too can buy my book titled Hands-On Mobile and Embedded Development with Qt 5! It has a nice image of glacier ice on the cover, which I thought was appropriate for a technology founded in Norway and then continued in Finland.



https://www.packtpub.com/application-development/hands-mobile-and-embedded-development-qt-5

A big thanks to the co-founder of Trolltech, Eirik Chambe-Eng, who was gracious enough to write the forward at the last second. Tons of thanks to all the editors who also worked on this book.

One of the things I learned writing this book is that Qt is big. I already knew that, but now it's plainly apparent just how big it has grown. Not only are there major companies developing products with Qt, but it has a lot of different functionality and is not just about desktop widgets. There are a huge number of classes to cover.

You can check out the table of contents if you want to see what is covered and included. One area that I did not include is OpenGL ES. This is a huge topic and easily a book on it's own. It's something I would like to know more about, which is why I did not feel qualified to cover it. You need to know OpenGL Shader Language (GLSL), and I did not have the time to discover that to any real depth.

I hope that I covered topics that are relevant for mobile, embedded and IoT developers. From QtWidgets, QtQuick, QtSensors (of course) to In-app purchasing and building an embedded system with Qt Company's Boot To Qt and Yocto. I also explore Qt's newest platform - Qt for WebAssembly, which allows you to serve Qt applications from a web server to run in a web browser.


Enjoy!

The Future of AST-Matching refactoring tools (EuroLLVM and ACCU)

I recently made a trip to LLVM in Brussels and ACCU in Bristol. It was a busy week. I gave a talk at both conferences on the topic of the future of AST Matchers-based refactoring.

As usual, the ‘hallway track’ also proved useful at both conferences, leading to round-table discussions at the LLVM conference with other interested contributors and getting to talk to other developers interested in refactoring tooling at ACCU.

Presentations

The learning curve for AST-Matcher-based refactoring is currently too steep. Most C++ developers who are not already familiar with the internal Clang APIs need to invest a lot in order to learn how to make such bespoke tooling to improve and maintain their codebase.

The presentations both include demos of steps I’ve been taking to try to address these problems.

The first demo is of clang-query discovery features which aim to reduce the need to infer AST Matcher code by examining the Clang AST itself. I also showed the debugging features I am preparing to upstream to clang-query. Finally – in terms of demo content – I showed a Qt tool which can eliminate some of the additional difficulties and problems of long-developer-iteration-time.

The debugging features and the Qt tool were world exclusives at the LLVM conference (and at the ACCU conference because most people didn’t attend both 🙂 ). I hadn’t shown them to anyone else before, so I was quite happy the demos went well.

Videos

My 25 minute presentation to the LLVM developers tried to show that these changes can make mechanical refactoring more easily available to C++ developers.

The aim was to show the features to the LLVM community to

  1. illustrate the issues as I see them
  2. get some feedback about whether this is a good direction
  3. introduce myself for the sake of further code reviews (and collaborators). As this was my first LLVM conference, I am not already familiar with most of the attendees.

My 1.5 hour ACCU presentation is a far-less-rushed presentation of the same tools and a repetition of some of the content at code::dive 2018. In the ACCU presentation, the new demo content starts about 30 minutes in. This talk is the one to watch if you are interested in using mechanical refactoring on your own code.

Feedback was very positive from both talks, so I’m happy with that.

Qt Tooling

Earlier this year I refactored the clang AST dump functionality. It was previously implemented in one class, ASTDumper, which had the dual responsibilities of traversing the clang AST and creating a textual representation of it. I separated the textual output from the generic output independent traversal, which introduces the possibility of alternative output formats such as JSON.

Of course, given my KDE and Qt contribution history, I would only create a generic tree traversal class in order to implement QAbstractItemModel for it.

The demos show all of the features you would expect from a point-and-click refactoring tool including exploring, feature discovery, debugging with in-source feedback, live source updates, experimental refactoring etc.

Of course, all of this requires changes to Clang upstream (for example to add the debugging interface) which was the point of my visit to EuroLLVM. Hopefully, once enough of the changes are upstream, I’ll be able to open source the tool.

The idea as always is to hopefully have enough functionality in Clang itself that IDEs such as Qt-Creator, KDevelop and Visual Studio would be able to integrate these features using their own GUI APIs, making the simple tool I made obsolete anyway. I only made it for demo purposes.

This will take the mechanical refactoring workflow which is currently

and turn it into

You will still do the same things, but with much faster development iteration to achieve the same result.

There is even more that can be done to make the process of mechanical refactoring with clang easier and faster. We discussed some of that at EuroLLVM, and hopefully all the pieces can come together soon. Meanwhile I’ll be upstreaming some of this work, talking at NDC Oslo, and at my local meetup group on this topic.

KDAB now offers standalone on-site Git training

New on-site Git training

KDAB’s training portfolio now includes a dedicated on-site training class about the version control system Git. The Git training material has been a supplemental chapter in our other on-site training offerings for quite a while. Given the recent rise in interest in this chapter we’ve reorganized, updated and extended the course. It now covers the latest changes in Git and most recent Git workflows out there.

This one-day training class provides a comprehensive introduction into modern software development with the code version control system Git. Beginning with the basic concepts, all the fundamental topics to manage code bases in Git will be covered.

Building on that, it expands into strategies for common workflows in software development life cycles and introduces options for collaboration on large scale or widely distributed projects.

Course Contents

  • Initial setup and checkout of Git repositories
  • Managing and committing changes
  • Navigating through Git history
  • Branching and merging
  • Development strategies
  • Collaboration via Gerrit code review and similar systems

View more details about the training course, including the full table of contents.

Another good reason to learn the basics of Git is that Qt itself is available through Git. A basic knowledge of Git is therefore essential if you want to apply patches back to Qt or try out a not yet released version of Qt.

The Git training course is completely platform-agnostic: we have already had several training sessions including people working with either Linux, macOS or Windows. We’re focusing on Git concepts explained by Git command-line usage which is applicable on any operating system. If needed we can also give a short introduction in graphical user interfaces for Git, such as TortoiseGit, Git extensions, git-gui, etc..

This training course has a duration of one day and is currently offered as on-site training only. It is suited for developers who work on software collaboratively or who need to maintain a delivered/deployed version while simultaneously working on new features for the next one.

Contact us

If you’re interested in a Git training course, please contact us!

The post KDAB now offers standalone on-site Git training appeared first on KDAB.

Qbs 1.13 released

We are happy to announce version 1.13.0 of the Qbs build tool. This is the last version to be released under the auspices of the Qt Company, but certainly not the least.

Highlights

Transparent pkg-config support

Qbs projects can now make use of pkg-config modules. Syntax-wise, the same dependency mechanism as for Qbs’ own modules is used. For instance, on a typical Linux machine with an OpenSSL development package installed, the following is enough to let a Qbs project build against it:

Depends { name: "openssl" }

Internally, this functionality is implemented on top of module providers, a powerful new feature that allows creating Qbs modules on demand.

Automatic Qt detection

Projects with a Qt dependency now set up the required Qt-specific modules (such as Qt.core) automatically at build time, using either the PATH environment variable or the moduleProviders.Qt.qmakeFilePaths property. It is therefore no longer strictly necessary to create a profile using the setup-qt command. Behind the scenes, this feature is also implemented using module providers.

Job Pools

The global limit for the number of concurrently running jobs does not make sense for all types of commands. For instance, linkers are typically I/O-bound and take up a lot of memory, so it often makes sense not to parallelize them as much as compilers. Here’s how job-specific limits are set on the command line:

$ qbs --job-limits linker:2,compiler:8

These limits can also be provided via preferences and in project files. The details are explained here.

What else is new?

Language Improvements

Rules are no longer required to specify output artifacts. As a result, rules whose main purpose is in their “side effect” will look more natural now, as they don’t need to declare a dummy output file anymore.

It is no longer necessary to start each project file with an “import qbs” line.

The Application, DynamicLibrary and StaticLibrary items have new properties install and installDir for more convenient installation of target binaries.

We introduced Process.atEnd() and FileInfo.canonicalPath().

C/C++ Support

GNU linker scripts are now scanned recursively to catch changes to included linker scripts. Thanks to Ola Røer Thorsen for this contribution!

The new cpp.linkerVariant property allows to force the use of ld.gold, ld.bfd or lld for linking.

Qt Support

We introduced the new property Qt.core.enableBigResources for the creation of “big” Qt resources.

Static builds now pull in the default set of plugins as specified by Qt, and the user can specify the set of plugins by type.

Android Support

The AndroidApk item has been deprecated. Instead, a normal Application item can and should be used now.

Building Qt apps is properly supported now. Some small changes to Qt’s androiddeployqt tool were necessary to achieve this, so you need at least Qt 5.12.

Autotest Support

There is an autotest module now, which you can use to specify arguments and working directory per test.

Various things

We introduced the texttemplate module, a facility similar to qmake’s QMAKE_SUBSTITUTES feature.

We added basic support for Google Protocol Buffers (for C++ and Objective-C). Thanks to Ivan Komissarov for this contribution!

Try It!

The Open Source version is available on the download page, and you can find commercially licensed packages on the Qt Account Portal. Please post issues in our bug tracker. You can also find us on IRC in #qbs on chat.freenode.net, and on the mailing list. The documentation and wiki are also good places to get started.

Qbs is also available on a number of packaging systems (Chocolatey, MacPorts, Homebrew) and updated on each release by the Qbs development team. It can also be installed through the native package management system on a number of Linux distributions including but not limited to Debian, Ubuntu, Fedora, and Arch Linux.

Qbs 1.13.0 is also included in Qt Creator 4.9.0, which was released earlier this week.

So, what now?

Preparations for handing over the project to the community are ongoing. Stay tuned for further announcements.

The post Qbs 1.13 released appeared first on Qt Blog.

Modernizing SCADA HMIs

Supervisory control and data acquisition (SCADA) systems have been around since the 1950s, far longer than most other types of computer applications. Their rock-solid performance has been responsible for the streamlining of any industry that needs precise and consistent controls: building automation, energy management, part machining, printing and packaging, robotic assembly, ship building, water treatment, woodworking, and many more. However, this long legacy can also carry a hidden drawback – the user interfaces of many SCADA devices look more appropriate as part of Windows for Workgroups than the modern age.

This situation is ripe for change. Now that everyone carries superior user interfaces in their pocket at all times, even the non-designers responsible for running the system expect their SCADA human-machine interface (HMIs) to have a certain level of polish and sophistication. Having implemented attractive SCADA HMIs for our customers, we believe that Qt is the right tool to build the modern SCADA system.

Download the whitepaper and read more…

The post Modernizing SCADA HMIs appeared first on KDAB.