AI: How machines learn to ‘see’

How can sheet metal machinery learn to ‘see’ using artificial intelligence (AI)? In the December 2024/January 2025 issue of ISMR, we track the AI computer vision journey of sheet metal specialist, TRUMPF (its headquarters in Ditzingen, Germany, pictured above).

===

TRUMPF is a high-tech company, based at Ditzingen (Germany), offering manufacturing solutions in machine tools and laser technology. It drives digital connectivity in manufacturing through consultancy, platform products and software. It is one of the technology and market leaders in versatile machine tools for sheet metal processing and industrial lasers.

The company is continuing to expand its activities in artificial intelligence (AI), with its vision to become “the leading user and leading provider of AI solutions in our industry in five years’ time.” TRUMPF has been strengthening and coordinating its AI activities with a new competence centre to make internal processes more efficient, with the help of AI. The sheet- metal specialist recognises AI’s potential for increasing productivity in, for example, software development and administrative areas. It also wants to bring more AI innovations to market for its customers in the machine tool, laser technology and electronics sectors.

TRUMPF already offers its customers solutions that work with AI. In machine tools, for example, its AI solutions support users in sorting cut components or improving the design of assemblies. In laser technology, TRUMPF’s AI solutions help with process control. This reduces disruptive influences such as contamination or scratches on a component. On the hardware side, TRUMPF already supplies key components for expanding the computing capacity for AI developments with EUV and TRUMPF Photonic Components.

This article chronicles the company’s AI journey as it teaches its machinery to ‘see’ and fully embraces the technology for itself and its customers.

Vision applications for AI

The TRUMPF computer vision team is teaching artificial intelligence (AI) to classify data correctly. The specialists are willing to do whatever it takes, including categorising 100,000 images by hand.

“Sheet metal is an absolute nightmare for cameras!” said Korbinian Weiß. He has spent years trying to make the two things more compatible. “Sheet metal is thin and bendy, it reflects light, can be wobbly and comes in endless shapes and sizes,” he said. Cameras can struggle to even recognize a metal surface, but artificial intelligence (AI) is now offering new solutions to these problems.

Weiß heads up the AI image-recognition team at TRUMPF in Ditzingen. His preference would be to have at least 24 cameras installed in machines such as the TruLaser Center 7030, TRUMPF’s first fully automated laser cutting system. But, for the moment, the system has 12 cameras which deliver huge quantities of images and video clips.

These, and many other types of data, provide the key to continuously improving existing machines and developing entirely new products. This data forms the foundation of the computer vision team’s work.

Data meets AI

But what role does AI play in the TruLaser Center 7030 as it slices its way through one sheet after the next? And how do we define AI in this context, anyway?

AI is a research field with numerous subspecialties, one of which is machine learning. For machines, such as the TruLaser Center 7030, to “learn” – in other words, to get better and more efficient at what they do – they need tools and methods in the form of appropriate software. Computer vision is one of those methods.

“Ninety-five per cent of the solution is data, and just five per cent is AI,” confirmed Weiß, explaining where the team had to focus its attention. “The challenge was collecting the data in the first place, curating and labelling it, compiling datasets to meet different problem-solving goals and getting the right balance within the data.”

Sometimes traditional algorithms are all that are needed to solve a customer’s problem, but often it takes more. A lot has happened in computer vision since TRUMPF launched its Sorting Guide in 2020. As well as improvements in the technology and algorithm databases, there has also been a shift in people’s mindset.

“Nowadays, we’re thinking about data from the moment we start developing a product,” said Weiß. That is why the TruLaser Center 7030 contains cameras and also why TRUMPF now has access to entirely new business models, one of which is ‘Pay Per Part’.

Sorting Guide solution

Thirty-seven-year-old Weiß trained as a mechanical engineer, but he began developing software-driven initiatives not long after joining TRUMPF. One of the projects that he led was the TRUMPF Sorting Guide, which did not originally include AI in its plans. The project team initially thought the Sorting Guide would work on the basis of conventional algorithms.

“Everything seemed great in our test facility; the results were fantastic,” said Weiß. But then he took it to the test customer “and nothing worked”. The problem was the lighting. The algorithms were overwhelmed by the mix of light and dark materials, the reflective surfaces and all the objects in the visual environment.

“Without AI, we wouldn’t have been able to cope,” said Weiß.

To train the AI, the computer vision team had to manually classify and label 100,000 images. By telling the system whether a sheet metal part was visible or not in each image, and using the corresponding algorithms, the team was able to teach the software to continuously improve the accuracy of its predictions in multiple training loops.

Remote night shifts

In this business model, TRUMPF’s fully automated flagship is based at the customer’s site, where it produces the required parts. However, control of the machine lies in the hands of a TRUMPF team at the Neukirch site in Saxony (Germany), which operates in three shifts (including at night).

The cameras keep the team updated on every aspect of the machine’s operation and deliver a non-stop stream of data. If a sheet metal part gets stuck, the cameras record a short clip starting a few seconds before the event and ending a few seconds after. This helps the AI to learn how to avoid such errors in the future.

AI offers numerous potential benefits and uses, some of which are still in their infancy. Examples include more efficient machine utilisation; longer running times; higher part quantities; savings on materials; predictive maintenance and assistance systems.

“There’s a lot happening in computer vision at the moment,” said Weiß. He highlighted TRUMPF’s 2024 activities, and its introduction of features such as smart cameras (some with AI functions) to TRUMPF machines.

A call to action

Professor Dr. Sepp Hochreiter is regarded as one of the world’s pre-eminent experts on artificial intelligence. Originally from Bavaria, he heads the Institute for Machine Learning and the Artificial Intelligence Laboratory at Johannes Kepler University Linz. Last year, he received the German AI Award from the Axel Springer Group’s news brand Die Welt, the most generous award of its kind in Germany. His current projects include his own language model.

To read the rest of this article in ISMR’s December 2024/January 2025 issue, please see https://joom.ag/Vg7d/p38