TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
Spherical CNNs on Unstructured Grids
Convolutional Neural Networks on non-uniform geometrical signals using Euclidean spectral transformation
QuadriFlow: A Scalable and Robust Method for Quadrangulation
3DLite: Towards Commodity 3D Scanning for Content Creation
6-DOF VR Videos with a Single 360-Camera
Unsupervised Texture Transfer from Images to Model Collections
Real-time Interactive Tree Animation
Automatic Thumbnail Generation Based on Visual Representativeness and Foreground Recognizability
A surface approximation method for image and video correspondences
Stereo Panorama Video Stitching
RGB-D SLAM and Real-time Reconstruction
Research Statement
My research interests are computer graphics, computer vision, virtual reality and geometry processing. My goal is to provide ready-to-use scanning geometry and offer high-level understanding to the 3D data.
Skills
Proficient with C/C++ Programming
Good at Python/Matlab/CUDA/GLSL/HLSL GPU Programming
Familiar with Pytorch/Tensorflow
Familiar with Java/C# Programming
Familiar with PHP/Javascript/Unity/Android programming
My Mission
To my knowledge, there are creative ideas and awesome applications emerging every year, and the demos are very fancy. However, most of these potential applications can hardly be used in common days, mostly due to the problem of robustness in graphics or poor accuracy in vision. I view my mission as to solve problems in computer graphics and vision, particularly in environment reconstruction from images and video.
Recently, my proposed solution is to reconstruct and quantize the 3D data as CAD models and textures on top of that. I have solved the problem of surface parametrization to a industry useable level (quadrangulation, see my QuadriFlow project), and will use this as a foundation to solve the 3D texture generation. Hopefully, I will find practical solutions and bring them to market in the future.
Publications
Adversarial Texture Optimization from RGB-D Scans
Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kudu, Chiyu "Max" Jiang, Leonidas Guibas, Matthias Niessner, Thomas Funkhouser
CVPR 2020
Joint texture optimization with a learned deep metric which is robust to complex scanning errors.
Learning Local Implicit Grid Representation for 3D Scenes
Chiyu "Max" Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Niessner, Thomas Funkhouser
CVPR 2020
Learning a chunk-based deep SDF for scene reconstruction.
NeurVPS: Neural Vanishing Point Scanning via Conic Convolution
Yichao Zhou, Haozhi Qi, Jingwei Huang, Yi Ma
NeurIPS 2019
Deep learning-based vanishing point detection method: Warp the image into conic space and use CNN to evaluate corresponding potential vanishing point.
FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image
Jingwei Huang, Yichao Zhou, Thomas Funkhouser, Leonidas Guibas
ICCV 2019
We introduce the novel problem of identifying dense canonical 3D coordinate frames from a single RGB image. We observe that each pixel in an image corresponds to a surface in the underlying 3D geometry, where a canonical frame can be identified. We propose an algorithm to predict it from RGB.
TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Niessner, Leonidas Guibas
CVPR 2019 (Oral Presentation)
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e.g., color texture maps). The key idea is to utilize a 4-rotational symmetric (4-RoSy) field to define a domain for convolution on a surface.
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, Leonidas Guibas
CVPR 2019 (Oral Presentation)
Estimate 6DOF 3D object pose with unknown categories from 2D images. We create the synthetic dataset by rendering canonical object coordinates to images and learn it from neural networks.
Spherical CNNs on Unstructured Grids
Chiyu 'Max' Jiang, Jingwei Huang, Karthik Kashinath, Prabhat, Philip Marcus, Matthias Niessner
ICLR 2019
We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals.
Convolutional Neural Networks on non-uniform geometrical signals using Euclidean spectral transformation
Chiyu 'Max' Jiang, Dequan Wang, Jingwei Huang, Philip Marcus, Matthias Niessner
ICLR 2019
We develop mathematical formulations for Non-Uniform Fourier Transforms (NUFT) to directly sample nonuniform data signals of different topologies defined on a simplex mesh into the spectral domain with no spatial sampling error.
QuadriFlow: A Scalable and Robust Method for Quadrangulation
Jingwei Huang, Yichao Zhou, Matthias Niessner, Jonathan Shewchuk, Leonidas Guibas
Symposium on Geometry Processing, 2018 (Best Paper Award)
Robustly and Efficiently Convert Triangle Manifold Meshes to High-quality Quadrilateral Manifold Meshes. It approximates the mixed-integer optimization problem with a maximum flow problem.
3DLite: Towards Commodity 3D Scanning for Content Creation
Jingwei Huang, Angela Dai, Leonidas Guibas, Matthias Niessner
SIGGRAPH Asia, 2017
Complete the geometry and color of 3D scanning model. Detect planar structure of the geometry, extrapolate to complete the scene. Texture optimization by fixing misalignment, motion blur and auto-exposure artifacts.
6-DOF VR Videos with a Single 360-Camera
Jingwei Huang, Zhili Chen, Duygu Ceylan, Hailin Jin
IEEE VR, 2017
Build a VR application that offer 6-DOF freedom with translation and rotation with input as a monoscopic 360 video.
Unsupervised Texture Transfer from Images to Model Collections
Tuanfeng Y. Wang, Hao Su, Qixing Huang, Jingwei Huang, Leonidas Guibas, Niloy J. Mitra
SIGGRAPH Asia, 2016
Transfer textures from product images to 3D shapes. The increased texture variation in ShapeNet is validated to be effective for RenderForCNN. The technique can future be used in improving texture of scanning data.
Real-time Interactive Tree Animation
Ed Quigley, Yue Yu, Jingwei Huang, Winnie Lin, Rod Fedkiw
IEEE TVCG, 2017
Posing and Animating botanical tree models interactively in real time. We use an articulated rigid body model with as-stiff-as-desired rotational springs in conjunction with our newly proposed simulation technique.
Automatic Thumbnail Generation Based on Visual Representativeness and Foreground Recognizability
Jingwei Huang, Huarong Chen, Bin Wang, Stephen Lin
ICCV, 2015
Generating image thumbnail balancing between visual representativeness and foreground recognizability. We propose a combination of various features and use SVM model to learn good thumbnail.
A Surface Estimation Method for Space-time Correspondences
Jingwei Huang, Bin Wang, Pradeep Sen, Wenping Wang
IEEE Transactions on Image Processing, 2015
Using B-Spline Approximation and Extension method to refine dense non-rigid correspondences in images or video.
Other Selected Projects
Real-time RGB-D SLAM and 3D Reconstruction
Jingwei Huang, Matthias Niesser, Leonidas Guibas
In progress
We use plane matching, structural information together with SIFT feature to do real-time RGBD SLAM. The system can get globally consistent structure in a large space with a long scan. Currently, I am trying to use planes to abstract the scenes, and using texture synthesis to achieve a ready-to-use 3D reconstruction.
Stereo Panorama Stitching with 14 GOPROs
Jingwei Huang
Qingxian Tech.
This is what I do as a startup during 3/2015 and 8/2015. The company was purchased by Wei-Shi-Zai-Xian in 3/2016.
PhysBowling: Real-Time Interactive Simulation of Rigid Bodies
Jingwei Huang, Bin Wang
Tsinghua
Self-implemented Rigid Body Physics Engine from Scratch.
GPU Acceleration of Incompressible Smoothed Particle Hydrodynamics and Applications
Jingwei Huang, Yun Fei, Bin Wang
Tsinghua
Using DirectX Parallel Computing (HLSL) to implement a real-time SPH Solver for Water Simulation.
GPU Ray Tracing
Jingwei Huang, Bin Wang
Tsinghua
Using OpenGL Parallel Computing (GLSL) to implement a fast GPU ray tracer.
