OpenSVBRDF released: a large-scale 6-dimensional material real-shot database at the ImageNet level in the material world

In the field of computational graphics, the appearance of a material describes the complex physical interactions between real objects and light. This description is often called a Spatially-Varying Bidirectional Reflectance Distribution Function (SVBRDF). It is an integral component in visual computing and has been widely used in fields such as cultural heritage, e-commerce, video games, and visual effects

over the past two decades , especially after deep learning became popular, the demand for high-precision and diverse digital material appearances in academia and industry has continued to increase. However, due to technical challenges, collecting large-scale databases is still a very difficult task. Currently, the number of publicly available material appearance real-shot databases is very limited

For this reason, Zhejiang University The research team from the National Key Laboratory of Computer-Aided Design and Graphics Systems and Hangzhou Xiangxin Technology Co., Ltd. jointly proposed a new integrated system for robust, high-quality and efficient acquisition of planar anisotropic material appearance. Using this system, the research team built the OpenSVBRDF public material database.

The following is a display of some material samples in the OpenSVBRDF database, as shown in Figure 1. Each row belongs to the same material category

This is the first large-scale measured database of 6-dimensional SVBRDF, with a total of 1,000 high-quality plane samples and a spatial resolution of 1,024×1,024. Equivalent to more than 1 billion measured BRDF, covering 9 categories including wood, fabric and metal.

Database home page: https://opensvbrdf.github.io/

Currently, the database is completely free for non-commercial applications. You only need to submit basic information on the website to apply for an account. After review, you can directly download relevant data and codes including GGX texture maps. The related research paper "OpenSVBRDF: A database containing measured spatially varying reflectivity" has been accepted by the top international conference on computer graphics ACM SIGGRAPH ASIA 2023 (journal paper)

Please click the following link to view the paper homepage: https://svbrdf.github.io/

Technical Challenges

If the original meaning is changed, the content that needs to be rewritten is: According to [Lawrence et al. 2006], the direct sampling method is to make dense measurements of the physical material under different lighting and viewing angles. Although this method can obtain high-quality and stable acquisition results, it is less efficient and requires a lot of time and storage costs. Another option is a prior knowledge-based reconstruction method, which can reconstruct materials from sparse sampled data. Although this method improves efficiency, the quality of reconstruction may be unsatisfactory when the prior conditions are not met [Nam et al. 2018]. In addition, although the current optical path multiplexing technology has achieved high acquisition efficiency and reconstruction quality, the robustness of the algorithm still needs to be improved in processing highly complex materials, such as brushed metal and polished veneer [Kang et al. 2018]

Figure 2: Representative work of existing material collection research. From left to right they are [Lawrence et al. 2006], [Nam et al. 2018] and [Kang et al. 2018]. Among them [Kang et al. 2018] is the team's early work published in ACM SIGGRAPH in 2018.

Hardware

In order to efficiently scan the material appearance, the research team built a near-half-cube near-field illumination multiplexing device. The size is approximately 70cm×70cm×40cm. The sample is placed on a clear acrylic plate and can be quickly slid in/out via drawer slides to achieve high throughput rates. The device consists of 2 machine vision cameras and 16,384 high-brightness LEDs. The two cameras capture samples from angles of approximately 90 degrees (primary viewing angle) and 45 degrees (secondary viewing angle). The LEDs are distributed on 6 sides of the device. The self-developed high-performance control circuit is responsible for independent brightness control of each LED and achieves high-precision synchronization of light source projection and camera exposure at the hardware level.

Please see Figure 3: The appearance of the acquisition equipment and the photos taken from two different angles

Acquisition and reconstruction

This system uniquely combines two currently popular methods, network-based prediction and fine-tuning, to improve the efficiency of physical acquisition. Optimization of differentiable lighting patterns while further improving the quality of results through fine-tuning. This is the first time to achieve high robustness, high quality and high efficiency of acquisition and reconstruction of planar SVBRDF

Specifically, in order to reconstruct the physical sample, the researchers first achieved this by Match dense SIFT features to establish high-precision correspondence between two camera views. For physical acquisition, the lighting pattern is first optimized as part of an autoencoder to achieve efficient acquisition. The autoencoder automatically learns how to reconstruct complex appearances based on measurements from two views and represents the results as intermediate neural representations. Subsequently, the neural expressions were fine-tuned by plotting image errors to improve the quality and robustness of the final results, based on photos taken by a principal-view camera under 63 equivalent linear light sources. Figure 3 shows the processing flow of the entire system. For details, please refer to the original paper

Figure 4: Acquisition and reconstruction process of the entire system.

Results

The research team collected appearance data and collected a total of 1,000 samples divided into 9 categories. To make it easier to use the physics-based rendering pipeline (PBR), the study also adapted neural expressions to industry-standard anisotropic GGX BRDF model parameters. Figure 5 shows the sub-parameters and properties of the material reconstruction results. Each sample contains 193 original HDR photos (total size 15GB), intermediate neural expressions (290MB), and 6 maps, including texture maps and transparency maps representing GGX parameters (total size 55MB). The spatial resolution of both neural expressions and texture maps is 1,024×1,024

After rewriting: Figure 5: Breakdown of reconstructed material results Properties (including neural expression, diffuse reflectance, specular reflectance, roughness, etc.)

In order to prove the correctness of the reconstruction results, the researchers took the photos from the main perspective (the picture below) First row) and neural expression mapping results (second row below) are compared. Quantitative errors (expressed as SSIM/PSNR) are noted at the bottom of the plot. As can be seen from the results in the figure below, this system achieves high-quality material reconstruction (SSIM>=0.97, PSNR>=34db).

Figure 6: Comparison of real photos and neural expression rendering results from the main perspective.

In order to further prove the generalization of the reconstruction results in the viewing angle domain, the researchers compared the photos taken from two viewing angles under the illumination of a point light source and the results drawn using GGX fitting parameters. The comparison verifies the cross-view correctness of the reconstruction results.

Picture 7: Comparison of actual photos taken and results drawn using anisotropic GGX fitting parameters in two viewing angles

The researchers also demonstrated the application of the database in three aspects: material generation, material classification and material reconstruction. Please refer to the original paper for specific details.

Figure 8 shows the process of using OpenSVBRDF to train MaterialGAN to achieve material generation and interpolation

Figure 9 shows the process of using OpenSVBRDF to train active lighting to improve material classification accuracy

Rewritten content is as follows: Picture 10: Using OpenSVBRDF to improve the quality of BRDF reconstruction based on single-point sampling (left) and optical path multiplexing (right)

Outlook

Researchers will work hard to expand the existing database and add material samples showing diverse appearances. In the future, they also plan to build a large-scale, high-precision measured object database that includes both material appearance and geometric shape. In addition, researchers will design a public benchmark in the direction of material estimation, classification and generation based on OpenSVBRDF, and provide solid data guarantee to promote the future development of related research through objective and quantitative standard testing.

The above is the detailed content of OpenSVBRDF released: a large-scale 6-dimensional material real-shot database at the ImageNet level in the material world. For more information, please follow other related articles on the PHP Chinese website!