All-in-one encoder/decoder approach for non-destructive identification of 3D-printed objects

1.   Introduction

Owing to recent advancements, the potential and applicability of three-dimensional (3D) printing have received considerable attention from industry and academia ^[1]. In the medical and manufacturing industries, domain experts can use 3D printing to model and test simulated prototypes before creating a final product ^[2]. Researchers and designers combine hardware and printing materials to iteratively design and build prospective models for new technologies ^[3]. In addition, end users can afford inexpensive 3D scanning and editing tools to create 3D objects for personal and professional use ^[4]. Therefore, 3D printing technology is expected to be widely used as an infrastructure for supporting industries and our daily lives.

Despite the accelerating developments in 3D printing, recent technological advances in 3D scanning and modeling have made original 3D objects vulnerable to illicit duplication. Therefore, whether a 3D object has been printed legally must be assessed ^[5,6,7]. Although 3D-printed objects can be identified by using two-dimensional barcodes and radio-frequency identification (RFID) tags, these identifiers can be easily removed or changed. Therefore, 3D-printed objects must be detected and authenticated in a non-destructive manner.

For non-destructive authentication, more advanced methods have been proposed to identify 3D objects by using external and internal sensing methods. An external sensing method extracts identification information from the surface of a 3D object and employs deep learning methods to improve its recognition accuracy ^[8,9]. With internal sensing methods, signal processing is required to detect internal information. A detection system consisting of a beam projector and a camera interprets an image reflected from a 3D object to recognize its internal area ^[10]. Acoustic sensing and terahertz (THz) systems detect internal information by transmitting and receiving specific frequencies to the object's interior, where the identification information is stored ^[11]. However, internal sensing methods are limited in identifying 3D objects that become partially broken during molding, whereas external sensing methods are limited to specific printing situations. Such identification systems are also inconvenient because the encoding and decoding processes for the identification information are not integrated well.

To overcome these challenges, we propose an all-in-one nondestructive 3D object identification method. The proposed method comprises three phases: 3D code insertion for stereolithography (STL), THz-based detection, and code extraction. During the 3D code insertion step, a one-dimensional barcode describing a 3D model's identification information is generated and inserted into the model before printing. During the detection step, the identification information is nondestructively detected using a single THz signal. Finally, the received THz signal is used to extract the 3D object's internal information. We implemented and tested the insertion, detection, and extraction procedures using a 3D graphic and THz time-domain spectroscopy (THz-TDS) system.

Our contributions are as follows. The first contribution is an all-in-one approach that combines both the insertions and detections of identification information using a THz-based detection system. This integrated method is able to insert the identification code within the 3D object and non-intrusively detects the code. The second contribution is a simple detection method based on a one-dimensional code that is appropriate to the detection system. With the use lengthier internal codes, the overall recognition time is improved by the THz system. The simple one-dimensional code allows the detection system to accelerate by using the one-line scan. The last contribution is the implementation and validation of the proposed all-in-on approach using a 3D graphic environment and the THz-TDS system. We evaluated our code insertion in a 3D graphic environment, and we evaluated the code detection in the THz-TDS system with different code types having different insertion depth and code width.

This paper is organized as follows. We first analyze previous studies in Section 2, and we introduce the all-in-one approach to inserting and detecting the identification information of a 3D object in Section 3. We then show the implementation and experimental results of the proposed method in Section 4 and conclude by discussing future work in Section 5.

2.   Related work

2.1.   External information detection

In general, 3D objects have been shown to prevent vulnerability to illicit duplication. A common approach is to attach physical tags such as 2D barcodes and RFID tags to 3D-printed objects. Although they may be convenient, these simple and inexpensive methods are also easily circumvented by removing or replacing the attached tags.

To address this problem, more complex methods have been proposed that do not require physical tags. The most common approach is an image-based surface recognition method for 3D objects with ^[12]. This method extracts information, known as features, from surfaces and stores it in a database for detection and tracking. A system known as PrinTracker ^[13] rescans the 3D-printed object to extract surface features that help identify printer types, methods, materials, and models. This detection and recognition method has recently been improved by employing deep learning algorithms ^[8] and using multi-scale keypoint-based feature fusion ^[14]. The multi-scale keypoint method detects a 3D object using voxelization and a convolutional neural network based on multi-scale keypoints from the point cloud input. A method for encoding patterns while minimizing external deformation has also been proposed ^[9]. Fine distortion is applied to the 3D model's surface, and it is detected and extracted using a camera to serve as an identification marker. In addition, a cloud service is connected to object detection in order to deal with a large number of 3D samples ^[15]. Furthermore, a method of applying a specific pattern onto the surface of an object has been proposed ^[16]. This method microscopically records an acoustic barcode on a surface, which allows allowing the object to be identified based on its response frequency. Because frequency is measured differently depending on the material and inserted code, this approach has the advantage of recognizing the frequency generated by a microscopic dent on the surface of the object. Although methods that use surface information as an identifier have the advantage of a relatively simple configuration, they are still susceptible to errors that result from deformation (i.e., direct or indirect damage) during the distribution process. Their application is also limited because identification information can manifest differently depending on the type of the printed object and the material is is made from.

2.2.   Internal information detection

Research based on various signal characteristics has been conducted to detect identification information inserted into 3D output. Aircode ^[10] is an image-based method that projects a pattern onto a 3D output and recognizes and extracts from the reflected image an empty space close to the surface of the 3D object. In addition, active acoustic sensing is performed to extract identification information from 3D objects by detecting the acoustic frequency of internal patterns that are configured in slicer software ^[11]. A specific structure is formed depending on the supporter and material characteristics of the output; this information is converted to a database and used for recognition. A method that uses THz waves for the identification of surfaces has also been studied ^[17,18]. THz waves are electromagnetic waves having a frequency between 0.3 and 3 THz and a wavelength between 0.1 and 1 mm, which is longer than that of visible or infrared rays; therefore, THz waves have the advantage of high transmittance in nonmetallic materials (such as plastics and wood). Based on these transmission and reflection characteristics, THz waves having a high transmittance through non-ionized materials are used to recognize the interior of 3D-printed objects and to obtain internal images. By utilizing features reflected from the surface, THz waves can recognize the unique features of an object ^[19]. Furthermore, InfraStructs ^[20] can extract images and information by scanning the interior of a 3D-printed object in one, two, or three dimensions. However, this approach necessitates the redesign and assembly of 3D output to insert the code if the code type is binary or matrix whereas the image and geometric code types require the scanning of a number of internal layers. This divided process and complicated scan make this approach difficult to deploy in the 3D printing industry. A recently introduced method detects the barcode of the surface of a 3D printed object based on THz ^[21]. This method enhances detection by using frequency-division multiplexing that simultaneously detects a barcode composed of parallel plates located on the surface of 3D printed model. However this method focuses on the surface of a 3D model, and requires further development to detect an internal code.

Each major identification method has advantages and disadvantages. External identification methods enable recognition through images, making them easier to implement, but vulnerable to errors caused by deformation. By contrast, internal identification methods have an advantage in the use of invisible information; however, they are more difficult to implement as they require a separate code insertion and detection steps or need to scan a number of internal layers. To avoid this problem, an invisible code must be automatically inserted during the output stage of the 3D printing process for future identification and recognition. For this purpose, we designed simple code patterns and automatic insertion process right before printing. Afterward, the code inserted within a 3D model simply and robustly detected in a non-intrusive way via THz-TDS system. The code is simple one dimensional code, the system only requires one line scan to a target 3D sample.

3.   Non-destructively inserting identification information and detecting it in a 3D-printed object

To identify 3D-printed objects, we introduce an integrated method for the insertion and detection of an identification code, as shown in Figure 1.

For the code insertion stage, we first design and generate a one-dimensional code for identifying a target 3D object to be printed, and we insert its corresponding 3D barcode into the STL file of the original model. Subsequently, the 3D model is printed with its identification code located internally and, invisible to consumers. The 3D object is then identified by using the THz system, which scans the THz signal reflected from its internal surface. Finally, the identification is reconstructed and matched with the original code in the database. The following subsections describe the design, insertion, and detection of the code.

3.1.   Internal code design and generation

An identification code inside a 3D object can be configured in a one-, two-, or three-dimensional form, according to a previous study ^[20]. However, as the number of dimensions increases, the recognition speed of the THz system decreases considerably; therefore, we configure the code in one dimension. One-dimensional barcodes are commonly used and have the advantage of being recognizable using a single scan. The information in our one-dimensional barcode is represented by 38-bit data, as shown in Figure 2. The three bits representing the category are used to classify the output, whereas the 16 bits that compose the identification number are used to identify the object.

Next, the one-dimensional code is converted into its corresponding 3D model and then inserted into a target STL file with a set of parameters. The insertion steps consist of checking parameters, finding an appropriate area, and generating and synthesizing a 3D model. First, each bit of the single-dimension code is mapped to a cube that is full if the bit is 1 and empty if the bit is 0. The width of each cube is decided based on the parameters inside the code to determine the appropriate area of the STL file. The resulting set of 3D cubes or the 3D barcode is positioned at the center of the widest point from the bottom of the target 3D object of the STL file. The depth of the barcode from the model's surface is determined based on the parameters of the code depth. To merge a 3D barcode into an object that will be printed out, the 3D barcode must be registered into a child node of the object to make an integrated single object. Before carving the 3D barcode into the object, the 3D barcode is configured as an empty polygon, because the 3D barcode is a model with a filled interior. To do this, the 3D barcode is registered as a child node and the sign of the 3D barcode's normal vector is reversed. Finally, the one-dimensional identification information is inserted into the 3D model of the STL file, and the modified STL is ready to print. The resulting 3D model is the same as the original model except for the internal area. The overall process is presented in Figure 3.

3.2.   Internal code detection based on THz

To detect the identification code located within a 3D-printed object, we use a THz signal that can be transmitted to and reflected from the object. As most 3D printing uses plastic materials, the use of THz is appropriate and covers a wide range of 3D printing applications. For this purpose, we design a THz-TDS system that sends and detects THz signals from a target 3D object. The THz-TDS system consists of a femtosecond pulsed laser, a beam splitter, a delay stage, a detector, an emitter, a sample stage and an interface for the PC, as shown in Figure 4. The THz signal generated by the femtosecond laser is split into two optical paths on the beam splitter. Then, the signal without delay is sent to the transmitter, whereas the signal with delay is synchronized with the received THz signal to produce a resulting signal. The PC is interlocked for data acquisition (DAQ), data processing, and control of the THz-TDS system. The THz-TDS system measures the THz wave reflected from a target sample while sending the THz wave pulse at the emitter with varying time delays. At this time, a strong signal is measured for pulses arriving at the detector along the same optical path, and a weak signal is measured when the THz pulses do not overlap each other. In this system, when a sample is placed in one of the optical paths, its characteristics are identified through the time delay at the measured signal intensity or maximum intensity value.

3.3.   THz sensing data processing and information extraction

We exploit a one-point scan method that sends and receives one pulse at every point of a horizontal line in regular intervals for code recognition based on a THz signal. The THz signal transmitted during the one-point scan generates a reflected wave depending on the composition of the layer in the 3D sample. The THz signals are reflected from the object's surface and from, the area that contains the object's internal data. To identify the data, we use the peak detection method based on a time range that corresponds to the data location. This time range is determined by the surface thickness and printing material of the object. For example, in Figure 5, the peak information at 210 ps is important for recognizing information located at a depth of 5 mm. The reflected signal is high if there is data or empty space, and the reflected signal is low if there is nothing or printing materiel. When this is applied to each pixel, all signals related to the 3D barcode are extracted. Subsequently, the corresponding identification code is regenerated by thresholding the extracted THz signal.

The detection accuracy must be improved to ensure recognition of a code located in deep areas. Although THz data analysis is possible through peak detection in a set of received THz signals from each pixel point, it is necessary to compensate for the weak signal received from a deeper depth and a narrower interval. For this reason, we apply an integral frequency of the fast Fourier transformation (FFT) by integrating a specific frequency range to include more signal information than the peak detection method. As depicted in Figure 6, for the THz characteristics in the 0.1–0.6T section for a sample having a data pattern width of 0.5–1.5 mm, the signal strength difference in the 0.1–0.3 THz region largely depends on the presence or absence of data. Therefore, the selection and integration of a specific range of THz frequencies are required to improve the detection of the internal code in small pixel intervals of less than 1 mm.

4.   Implementation and experiment

We implemented and tested the proposed all-in-one encoder/decoder approach for the nondestructive identification of 3D-printed objects. For code insertion, we implemented a 3D graphic module that generates barcodes by using Unity3D. The insertion module first imports the 3D model's STL file into the Unity3D environment and then creates and inserts a 3D barcode into the file. Finally, it synthesizes a 3D code by applying the original 3D model as a new STL file for printing. Figure 7 shows how the 3D barcode is generated and inserted into a 3D object.

We then implemented an identification code detection system based on the THz-TDS system. The implemented THz-TDS system consisted of a THz optics module, a stage, and an imaging PC. The THz optics system consisted of a Toptica femtosecond laser, delay stage, and DAQ device. The range of THz frequency of the femtosecond laser was from 0.1 to 3 T and its main component ranged from 0.3 to 1 T. We set the beam power to 13 mW to produce more efficient THz powers than 10 mW. The focusing distance from the emitter/detector to the sample was 30 mm, and its spot size was 1 mm. The stage used a THz emitter and detector and supported the x-y-z axis movement of a 3D target object. The imaging PC controlled the laser and received the resulting THz signal from the DAQ device. Figure 8 shows the proposed THz-TDS system.

First, we tested the all-in-one code generation and recognition processes. To do this, an identification code with an interval of 1 mm was inserted at a depth of 2 mm from the object's surface. Afterward, a new object with the internal code was obtained. The printed 3D target object is shown in Figure 9 (a model of the leaning tower of Pisa) and the area for data placement is in the lower part. A code is inserted inside the 3D printout and placed on the stage of the THz-TDS system. We scanned the internal code to confirm that it could be recognized. The barcode data were sensed using a one-point scan, and the code was extracted successfully without any problems. As shown in Figure 9, the code is located between 448 and 450 ps in the time domain; thus, the one-point scan method collects the reflected TH signal from that range. Finally, the data are extracted from each location, as illustrated in Figure 9.

We then evaluated the performance of the scanning identification code. To evaluate the performance of the code design, we selected one- and two-dimensional code patterns: a one-dimensional sequence of bars, and a two-dimensional area code, as shown in Figure 10. We used two codes of each type: the two-dimensional codes being 60 (6 $\times$ 10) and 40 (5 $\times$ 8) bits in size, and the one-dimensional codes being 38 and 28 bits in size. The four codes were printed using a PolyJet 3D printer that supports accurate printing based on fused deposition modeling (FDM). We placed each printed code pattern on the stage of the THz-TDS system and collected the reflected signals from the code.

We measured the overall time required to scan each code pattern. It is important to note that this scanning time is mainly related to stage control while the time required for detection and extraction is quite low or negligible. As shown in Figure 11, the experimental results indicated that two-dimensional barcodes take longer to scan than one-dimensional barcodes. The two-dimensional barcode scans required 45 s and more than 1 min for 45 and 60 bits, respectively, whereas the one-dimensional barcode scans required 15 and 30 s for 38 and 28 bits, respectively. Note that peak detection for the integral frequency was applied to improve detection quality when the 28-bit code was scanned, thereby leading to a relatively longer duration. Here, the scanning interval of the x-axis was 0.2 mm, and the THz signal from 0.1 to 1T was integrated to improve the detection quality. Testing for high-resolution and deep areas was found to take twice as long, as more precise scanning was required.

This result indicates that the proposed one-line scan is fast enough to support the 3D scan. Our scan took approximately 15 s at 2 bits/s, and the x-y scan of the random arrangement code took 2 min at 100 pixels/s ^[20]. The random arrangement method of InfraStuct inserted 100 blank spaces as data and needed to scan 178 layers to read the data. This wide scan also decreased the scan accuracy to 87%. However, our method uses one-dimensional code and requires a one-line scan. In spite of this analysis, it is still difficult to precisely compare scan time and accuracy due to the differently configured THz-TDS systems and the test environment.

5.   Discussion

Merging insertion and detection: Understanding how code generation and recognition are linked and integrated for practical use is important. Existing research has focused on the code detection step, which is not clearly connected to the code generation step. However, the proposed method integrates identification, generation, and detection to ensure that identification markers are invisible to consumers but can be conveniently managed by distributors and end users.

Single-dimension barcodes for efficient practical use: The potential of THz radiation is optimized, and a fast one-dimensional method is effective for practical use. Owing to the nature of 3D output, the configuration of one-dimensional and two- and three-dimensional information is possible. However, there may be variations in the internal structure during the printing process, and higher dimensions require more imaging time, which makes practical use difficult.

Depth and resolution of inserted code: As difficulties are caused by the depth and resolution, the code must be inserted as close to the surface as possible. In addition, to increase the usability of 3D-printed materials, THz must be incorporated into mixed-method imaging.

Repeatability and robustness: The proposed all-in-one approach achieved process integration with known information. The parameters used for code insertion such as depth, interval, and the number of bits were also used in detection and extraction. Therefore, the proposed approach is robust when fixed settings are used. However, the detection should be improved when there is no information about the code insertion step. In order to overcome this limitation, the detection should scan domain information longer in a target sample and select the appropriate depth and bit interval automatically based on a number of data samples.

Our method is subject to several limitations when inserting and detecting generated code geometries. Our method assumed that a sizable base area is sufficient for adding an identification code, because a sizable 3D model has a base or bottom area for standing. More elaborated processes should be explored for smaller samples or narrow objects. In addition, our method inserted data codes with geometries that form empty spaces in the filled space below, where the supporter was sliced and removed to form an empty space. The minimum width of the empty space ranged between 2 mm and 0.5 mm, therefore the probability of similar geometry is quite low. However, our code generation and detection could be further tested with different 3D printing environments and methods (i.e., digital ink writing), which is beyond the scope of this study.

6.   Conclusions

In this study, we proposed an all-in-one encoder/decoder approach for the nondestructive identification of 3D-printed objects. We introduced a general pipeline for the insertion and detection of the objects' identification data. We then designed a 3D barcode that represents a 3D-printed object and inserted it into the corresponding object before printing. We also introduced a THz-TDS system to detect and analyze the THz signal. Finally, the THz signal analysis module was used to extract the signal related to the code area and convert it to the original identification code. Through experimentation, we found that the one-dimensional barcode was efficient for practical use, and we tested the all-in-one approach by using a 3D-printed object. In addition, we demonstrated that the integrated FFT could recognize code patterns located in deeper areas.

As our work is the first step toward the non-destructive identification of 3D-printed objects, several research directions could be explored. First of all, it is necessary to automatically determine the resolution, depth, and peak position to extracting identification information. Our method used the parameters created from the insertion step and thus should be improved to make it as automatic as possible. Next, the integrated method consisting of code insertion and recognition should be improved to support diverse types of 3D objects. As the types and shapes of 3D objects vary, the code should be able to be inserted into any part of a 3D object. Last, more printing materials should be explored to support a wide range of printing methods. We hope that these improvements will help the distribution of 3D-printed objects while protecting the copyright of creators.

Acknowledgments

This work was supported by a National Research Foundation of Korea(NRF) grant funded by the Korea Government (MSIT) (No. 2021R1F1A1047682) and by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea Government (MSIP) (No. 2017-0-00417, Openholo library technology development for digital holographic contents and simulation).

Conflict of interest

All authors declare no conflicts of interest in this paper.