3D Visualization of thermal resistance and condensation problems using infrared thermography for building energy diagnostics

Abstract

Background

Building deteriorations instigated by material degradations or moisture intrusions are the primary causes for energy inefficiency in many existing buildings. For choosing appropriate retrofits, it is important to carefully diagnose and analyze building areas in need of improvements. In addition to reliable sensing and analysis of as-is energy performance, an intuitive recording and visualization of energy diagnostic outcomes are also critical to effectively illustrate the as-is building conditions to homeowners during retrofit decision-making processes.

Method

Toward this goal, this paper presents a thermography-based method to visualize the actual thermal resistance and condensation problems in 3D while taking static occlusions into account. First, several overlapping digital and thermal images are collected from the building areas under inspection. Using a computer vision method– consisting of image-based 3D point cloud and mesh modeling algorithmsactual 3D spatio-thermal actual 3D spatio-thermal models are generated where surface temperature can be queried at the level of 3D points. Based on the resulting 3D spatio-thermal models and by measuring the reflected and dew point temperatures, the actual R-values of building assemblies are calculated, and the condensation issues are analyzed. Taking static occlusions into account, (1) the distribution of the actual thermal resistance over each building assembly, (2) the detected building areas with condensation problems, and (3) the corresponding geometrical and thermal characteristics are jointly visualized within a 3D environment.

Results

To validate the method and investigate the perceived benefits, four experiments have been conducted in existing buildings. Surveys are also conducted by professional energy auditors. The proposed method provides 3D visual representation of the actual thermal resistance distributions and building areas associated with condensation issues at the level of 3D points across geometrical forms while taking static occlusions into account.

Conclusions

The experimental results and the feedback received from the professionals show the promise of the proposed method in facilitating systematic post-examination of building deteriorations and support retrofit decision-makings. Ultimately, converting surface temperature data obtained from an IR camera into 3D visualization of energy performance metrics and possible condensation problems enables practitioners to better understand the as-is building conditions.

Introduction

The emerging energy crisis of the building sector and the legislative measures on improving energy efficiency are steering the construction industry towards adopting new design concepts and construction methods that can decrease the overall energy loads. For example, in order to lower the energy cost required for space conditioning, building practitioners are embracing the concept of net-zero passive houses at the design phase. The use of building materials with higher thermal resistance is recommended for better insulation at the construction phase.

The problem of energy inefficiency is however not limited to the design and construction of new buildings. According to a recent report by the U.S. Department of Energy (U.S. DOE [2010]), around 35% of input energy in existing buildings in the U.S. is still being wasted. A primary source of these inefficiencies is thermal performance problems in the building envelope, which can cause an excess in energy consumption for space heating and cooling purposes. Roth et al. ([2005]) reported that such faulty behaviors can account for up to 10% of the total energy consumption in existing buildings during their operational phase. Despite their significance, identifying and assessing building performance problems and improving the energy efficiency under such conditions is not trivial.

Today, building practitioners widely use Infrared (IR) thermography to identify sources of energy loss in building envelopes. The IR thermography –which detects and measures heat variations from surfaces in a non-destructive manner– is particularly known for its usefulness for identifying thermal defects. Nonetheless, current thermographic inspection processes have several inefficiencies. For example, thermal images captured using consumer-level IR thermal cameras typically have low spatial resolutions (160 × 120 or 320 × 240 pixel) and small fields-of-view compared to their digital counterparts (>1 megapixel). As a result, sensing thermal performance data for buildings, even a single room, requires large numbers of thermal images (e.g., ~300 to 400 thermal images for around 4m × 5m room based on experiments). Considering significantly large collections of 2D thermal images that need to be manually analyzed for the purpose of assessing thermal conditions, their direct application for practical building diagnostics can be time-consuming and labor-intensive. In order to improve the process of sensing thermal performance using raw 2D thermal images, several studies (Borrmann et al. [2013]; [Ham and Golparvar-Fard 2013]; Lagüela et al. [2013]; Wang et al. [2013]) have recently focused on developing methods for 3D thermal modeling of building environments by leveraging digital photogrammetry or laser scanning, and promising results have been reported.

These methods for 3D thermal modeling provide a detailed representation of the actual temperature deviations on building surfaces, which is very effective for energy diagnostics. Nevertheless, the problem of characterizing and visualizing performance metrics and thermal problems from these models for post-inspection is still unexplored. The analysis of abnormal thermal regions – either from 2D thermal images or 3D thermal models – is still primarily based on the interpretation of the measured surface temperatures such as hot or cold spots. Yet, the surface temperatures alone do not directly represent the energy performance problems. Due to a lack of a benchmark, the interpretation of energy performance problems is still primarily based on the auditors' experiences and knowledge, which can cause the outcomes to be subjective or inconsistent. Considering a limited number of well-trained professionals, it is also likely that auditors' knowledge and experience would adversely impact the quality of the inspection.

Another important issue – which is often not considered– is that the deterioration rate of building assemblies typically vary even over small surface areas. For instance, when it comes to building envelopes, various regions across the surface may suffer from different levels of degradation. As such, their conditions would change differently over time. Therefore, a method for building diagnostics should assess and visualize the as-is building conditions at point-level across geometrical forms of the building elements so that defects can be precisely detected and localized.

The presence of static occlusions in existing buildings has also raised concerns about the applicability of thermography-based methods for building diagnostics. Ideally, for accurate measurement of surface thermal performance, any objects around the building interior surfaces need to be removed prior to thermographic inspections. However, in practice, for most in-use buildings, it is not trivial to take away all objects that block the line of sight due to the immobility of objects or space limitations. Theoretically, there is no way to measure the thermal performance for the building areas that are occluded by objects, except for encouraging practitioners to remove as many objects as possible before thermographic inspection. Thus, this issue remains as an open challenge.

To address these challenges, this paper presents a new interactive method for 3D visualization of the actual R-value of building assemblies and condensation problems. The proposed method builds upon the authors' prior computer vision research on 3D spatio-thermal modeling using large numbers of unordered thermal imagery collected by a consumer-level IR camera [(Ham and Golparvar-Fard 2013)]. Beyond synthesizing raw 2D surface temperature data, the method extracts and visualizes building energy performance indicators from the collected thermal imagery. It facilitates the process of identifying potential performance problems by allowing auditors to spend less time on analyzing large amounts of thermal images, and instead focus on investigating the causes of the problems and analyzing various retrofit alternatives.

The primary technical contributions of this paper are two-fold: (1) a method for 3D visualization of energy performance metrics and moisture issues at the level of 3D points across geometrical forms of building elements; and (2) a method for minimizing the adverse impact of static occlusions on thermographic inspection in real-world building environments. In the following sections, previous works on measuring R-values of building materials using thermal imagery and analyzing condensation problems are reviewed, and their challenges in applying to building condition assessment are discussed. Next, the research objective, underlying environmental assumptions, and the proposed visualization method are presented in detail. Finally, experimental results on four building assemblies in two case studies and the responses from domain expert evaluations are discussed.

Backgrounds

Thermal resistance (R-value) of building assemblies

Analyzing heat transfer conditions through building envelopes is now a required step in rating the energy performance levels of existing buildings (Fokaides and Kalogirou [2011]). This is because during the operational phase, the thermal resistance of building materials gradually decreases. Thus, the actual R-value of the building assemblies with thermal defects is typically lower than the notional value declared by their manufactures. Declining R-values of the building materials results in unnecessary heat transfer through building façades. This in turn increases the operational frequency of heating and cooling systems for space conditioning purposes. Hence, beyond using building materials with high R-value during the design and construction, it is important to explore how the as-is thermal resistance of building assemblies lowers over time.

Recently, several research groups have focused on the problem of non-destructive measurement of the actual heat transfer conditions of building assemblies using thermal images. Madding's work (Madding [2008]) is one of the earliest studies that propose a method for analyzing thermal images to estimate the actual heat transfer conditions for building envelopes. Based on the environmental assumption of a steady-state heat transfer condition of building environments, this work measured the R-value of drywall assemblies using the indoor surface temperature data obtained from thermal images. The measurements were experimentally conducted in controlled lab environments, and then they were compared with the theoretical expected properties. Likewise, based on the similar environmental assumptions, methods proposed in (Albatici and Tonelli [2010]; Dall'O' et al. [2013]; Fokaides and Kalogirou [2011]) measured the overall heat transfer coefficient of the building assemblies. For validation, several experiments were conducted for a few standard wall assemblies. In these studies, the difference between the measured values using thermal images and the notional values was reported to be in the range of ~10% (Madding [2008]), 10-20% (Fokaides and Kalogirou [2011]), and around 15% for cavity walls (Dall'O' et al. [2013]).

For quantifying the as-is thermal resistance of building assemblies, the current non-destructive method regulated by the international standard is to position heat flux meters (HFMs) –a transducer that produce an electrical signal proportional to the heat rate– on the representative building areas and directly measure the heat flow (International Organization for Standardization (ISO) [2007]). However, considering possible variations of the thermal conditions even over small surface areas, it is difficult to ensure that the thermal resistance measured by using a few HFMs will actually represent the entirety of an existing building envelope (Albatici and Tonelli [2010]; Dall'O' et al. [2013]). Interestingly, studies such as (Albatici and Tonelli [2010]) and (Dall'O' et al. [2013]) report 40-80% and 40-175% error between the measurements of the overall heat transfer coefficient using HFMs and their theoretical expected values. Although these observations are primarily based on limited experiments, when one considers the significance of these errors, the applicability of thermal imagery for quantifying the actual heat transfer condition of building assemblies becomes more attractive.

Despite such promising benefits of using thermography, the direct application of 2D thermal imagery as used in prior works is still challenging. In prior studies, a single or at most a few temperature data were extracted from the designated areas captured in 2D thermal imagery, and then a single measurement of the actual heat transfer condition was performed for each building assembly. Hence, these methods assumed that the obtained thermal resistance is representative of all spots on the inspected building assemblies. However, because of different deterioration rates, the actual heat transfer conditions may vary over small scales across geometrical forms. Thus, single measurements will not represent the dynamic variations in the actual R-values that are typically caused by non-uniform deteriorations in the building materials. More importantly, interactive 3D visualization of these dynamic variations in the context of the building under inspection will provide an opportunity for practitioners to better understand the as-is heat transfer conditions.

Condensation problems in building environments

In building environments, water vapor condensation happens wherever exposed building surfaces are at a lower temperature than the dew point. Typically, it is not easy to detect small amounts of the condensed water on building surfaces with the naked eye. Failure in detecting the onset of such condensation issues causes the condensed moisture to accumulate over time. Then, molds and dust mites may build up, which ultimately has potential for causing allergic reactions among the building occupants. In addition, the resulting wet surface may cause flaking paints and peeling wallpapers. This is when the building occupants are more likely to detect the problems with their naked eyes. In building environments, windowsills or thermal bridges around corners of the walls typically have low temperatures. Thus, energy auditors need to proactively analyze these areas whether the surface temperature has reached to the dew point.

Several studies have focused on analyzing the condensation phenomenon on building façades and the risk associated with the occurrence. Bellia and Minichiello analyzed the thermal and moisture performance of building assemblies, and focused on surface condensation issues of building façades and interstitial condensations within multi-layer walls (Bellia and Minichiello [2003]). Aelenei and Henriques investigated the external environmental conditions that cause condensation problems in building environments (Aelenei and Henriques [2008]). They concluded that the thermal convection effect and the amount of moisture in the air are the most influential factors in forming condensation problems on building façades. These prior works primarily examined the sources that cause condensation and identified ways that can reduce the risk associated with these problems. Nonetheless, devising energy auditing methods that can easily detect condensation problems in building environments is still an open research area. Today, by comparing the surface and dew point temperature, thermal cameras enable 2D detection of those building areas that suffer from condensation issues (Hoff [2011]). These detections are typically done on a single image displayed on the screen of the thermal camera. Because these images are not geo-tagged, it is often not easy to figure out at a later stage which building areas are associated with the detected condensation problems. Moreover, these 2D assessments do not provide any information about the surface area of the detected condensation problems. Hence, it is difficult at a later stage to quantitatively estimate the significance of their impact.

Overview of the proposed method

This paper presents and validates a 3D visualization method for quantitative assessment of the as-is building conditions. The method leverages a paired collection of digital and thermal images taken using a single IR thermal camera from the building environments under inspection, and accounts for static occlusions in the building environments. The hypothesis is that the quality of the current building condition assessment procedures can be improved with intuitive 3D visualizations that enable practitioners to (1) explore the distribution of the actual heat transfer conditions for building assemblies; and (2) detect condensation problems in 3D. The overall workflow is illustrated in Figure 1.

Figure 1

An overview of the data and process in the proposed 3D visualization method for building condition assessment using infrared thermography.

By using digital and thermal images, the proposed method first generates the 3D spatio-thermal mesh models which contain the surface temperature readings at the level of 3D vertices. Then, with environmental measurements, actual thermal resistances are calculated, and building surface areas suffering from condensation problems are detected at the level of 3D vertices in the mesh. Finally, taking possible static occlusions into account, the distribution of the actual R-value, the detected building areas with condensation problems, and the corresponding geometrical and thermal characteristics are interactively visualized within a common 3D environment. In the following sections, the underlying environmental assumptions, experimental setups, and each step of the proposed method are presented in detail.

Environmental assumptions and experimental setups

The proposed method is based on the following environmental assumptions: (1) the indoor building environment is assumed to have a quasi-steady-state condition of heat transfer during the thermographic inspection process. This is consistent with all recent works on non-destructive measurement of thermal properties (Albatici and Tonelli [2010]; Dall'O' et al. [2013]; Fokaides and Kalogirou [2011]; Ham and Golparvar-Fard [2014]; Madding [2008]). For achieving a quasi-steady-state condition of heat transfer, the building envelopes were not exposed to direct solar radiations and wind loadings as the inspections were conducted before sunrise on a non-windy and cloudy day. This minimizes the influence of the convective heat loss and the surface temperature rise on the exterior of building envelopes. As such, the possible increase in building surface temperatures caused by the release of the absorbed solar energy throughout a day was minimized; 2) it was further assumed that the heat transfer between the indoor surfaces and the thermal camera lens is due to thermal convection and radiation. This assumption is consistent with prior works (Fokaides and Kalogirou [2011]; Ham and Golparvar-Fard [2014]; Madding [2008]). In addition, the building interior spaces were heated for a few hours prior to thermographic inspections, which allowed to clearly capture the as-is heat transfer conditions through building façades during the winter season.

For testing the proposed 3D visualization and conducting professional surveys, several experiments were conducted on four wall assemblies and windows adjacent to the exterior in an existing residential building (Case #1) and an instructional facility (Case #2). For 3D spatio-thermal modeling, digital and thermal images were captured by using a FLIR E60 thermal camera that has a built-in digital camera. Each pair of digital and thermal images were captured simultaneously. The technical specification of the thermal camera used in experiments is presented in Table 1. When temperature deviations used for calculating thermal resistances (Measuring actual R-values at the level of 3D points) are too small, accurate measurement using inexpensive consumer-level IR cameras would be challenging. To alleviate this, similar to (Madding [2008]), the impact of the systematic error on the measurement accuracy was minimized by using a single device for measuring temperature differences.

Table 1 Technical specification of the thermal camera used for data collection

Method

Reconstructing 3D building thermal performance using 2D thermography

The first step to assess and visualize as-is building conditions at the level of 3D points is to model the actual thermal performance of the building under inspection in 3D. The method for generating 3D spatio-thermal models of building environments builds upon the authors' recently prototyped methods [(Ham and Golparvar-Fard 2013)] and (Golparvar-Fard and Ham [2014]). First, both thermal and digital imagery are collected by practitioners using a consumer-level IR camera. Then, the 3D geometrical point cloud of the building is automatically generated by using the collected digital images. Here, an image-based 3D reconstruction method is used, which consists of Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo (MVS) algorithms. The next step is 3D thermal modeling using the collected thermal images. Unlike digital images, thermal images are typically featureless in computer vision terms and have low spatial resolutions. The lack of distinct visual features impede the application of the same image-based 3D modeling algorithm used on digital images for 3D building geometrical modeling purposes. Instead, through a one-time process, the thermal camera is internally calibrated, and the relative pose of the thermal camera lens with respect to the digital camera lens on the IR thermal camera is estimated. Based on the obtained intrinsic and extrinsic thermal camera parameters, a 3D thermal point cloud model is generated using the dense reconstruction algorithm. The resulting 3D thermal and geometrical point cloud models are automatically superimposed, forming 3D spatio-thermal point cloud models. Next, the outcome is converted to a 3D thermal mesh model by using k-d tree structure and the nearest neighborhood searching algorithm. In this process, a geometrical baseline mesh is formed by using the boundary points extracted from the building geometrical point cloud as the basis for the transformation. The nearest 3D thermal point with respect to each vertex is searched for in the baseline mesh, and then the associated thermal values are mapped to the vertices. Figure 2 illustrates 3D thermal mesh modeling of two different building environments (top and bottom: Case #1 and #2 respectively).

Figure 2

3D thermal mesh modeling. (a): Unordered digital and thermal imagery, (b): Geometrical baseline mesh models, (c): 3D thermal mesh models of interior surfaces jointly represented with building geometrical point cloud.

Here, different from previous works [(Ham and Golparvar-Fard 2013)] and (Golparvar-Fard and Ham [2014]), and to better deal with static occlusions in building environments, a threshold is used to identify those thermal points that have a certain distance from the geometrical baseline mesh. It is noted that the thermal points may belong to objects on or around the wall, not the building assembly itself. Thus, when searching for the nearest thermal points to each vertex in the baseline mesh, a geometrical threshold based on the registration error of the point cloud and the baseline mesh is applied. At a later stage, the points from non-relevant objects are discarded and are not used for calculations. The details will be discussed in Section (Considering static occlusions in 3D visualization of the as-is building conditions).

Measuring actual R-values at the level of 3D points

The environmental measurements in this paper build on the environmental assumptions presented in the section (Environmental Assumptions and Experimental Setups), which is consistent with recent works on non-destructive measurement of R (or U)-value using thermography (Albatici and Tonelli [2010]; Dall'O' et al. [2013]; Fokaides and Kalogirou [2011]; Ham and Golparvar-Fard [2014]; Madding [2008]). Under such environmental assumptions, the overall heat transfer rate (dQ/dt) through a building surface with the area of (A), a thermal resistance (R), and a temperature difference of ΔT between inside and outside can be described using the following equation:

$$\frac{\mathrm{d}\mathrm{Q}}{\mathrm{d}\mathrm{t}}=\frac{1}{\mathrm{R}}\times \mathrm{A}\times \Delta \mathrm{T}$$

(1)

Here, the overall heat transfer (Q) in building environments can be described by the combination of thermal convection and radiation. First, the heat transfer caused by thermal convection between the interior surface of building environments and the indoor air is calculated using the following equation:

$${\mathrm{Q}}_{\mathrm{Convective}}={\mathrm{\alpha }}_{\mathrm{convective}}\times \mathrm{A}\times \left|{\mathrm{T}}_{\mathrm{inside},\mathrm{air}}-{\mathrm{T}}_{\mathrm{inside},\mathrm{wall}}\right|$$

(2)

Where α _convective is the convective heat transfer coefficient which is a function of the airflow (e.g., laminar or turbulent) and the temperature deviations between building surfaces and indoor air. Typical convective heat transfer coefficients of indoor building environments are within the range of 2–8 W/m²K. In this work, this value is adopted based on (Dall'O' et al. [2013]). As part of data collection, the inside and outside air temperatures, T_{inside, air} and T_{outside, air}, are measured using a thermometer. Next, the heat transfer between the interior building surface and the surroundings caused by thermal radiation is calculated using the following equation:

$${\mathrm{Q}}_{\mathrm{Radiation}}=\mathrm{\epsilon }\times \mathrm{\sigma }\times \mathrm{A}\times \left|{\mathrm{T}}_{\mathrm{inside},\mathrm{wall}}^4-{\mathrm{T}}_{\mathrm{inside},\mathrm{reflected}}^4\right|$$

(3)

Where ε is thermal emissivity and σ is Stefan-Boltzmann constant $\left(5.67\times {10}^{?8}\raisebox{1ex}{$\mathrm{W}$}\!\left/ \!\raisebox{-1ex}{${\mathrm{m}}^2{\mathrm{K}}^4$}\right.\right)$. For thermal emissivity, the measurement in (Mikron Instrument Company [2012]; Singham [1962]) was adopted. To measure the reflected temperature (T_{inside, reflected}), a small crumpled aluminum foil was placed on the inspection areas prior to the thermographic inspection (Figure 3). Since the crumpled aluminum foil has low emissivity and high reflectivity, the reflected temperature can be robustly measured from the crumpled foil (FLIR system [2010]). Prior works (Fokaides and Kalogirou [2011]; Madding [2008]) experimentally demonstrated that measuring the difference between surface and reflected apparent temperatures is one of the most sensitive factors on reliable R-value measurement in a non-destructive form. Here, instead of extracting a single temperature from the crumpled foil or averaging temperatures of a few target points as used in (Fokaides and Kalogirou [2011]; Madding [2008]), the average temperature is calculated from all pixels that represent the crumpled foil. To do so, the RGB color values of these pixels from the thermal image (Figure 3c) were converted into the corresponding absolute temperature values based on a normalized temperature spectrum.

Figure 3

Measuring reflected temperature. (a): crumpled aluminum foil located on the inspection surface, (b): thermal image simultaneously captured from the same viewpoint, (c): temperature readings on the crumpled aluminum foil surface.

It is assumed that interior surfaces in the given space under inspection have the same reflective temperatures, which is experimentally validated in (Fokaides and Kalogirou [2011]). Finally, by combining the Equations (1), (2), and (3), the thermal resistance can be described by using the following equation:

$$\mathrm{R}=\frac{\left|{\mathrm{T}}_{\mathrm{inside},\mathrm{air}}-{\mathrm{T}}_{\mathrm{outside},\mathrm{air}}\right|}{{\mathrm{\alpha }}_{\mathrm{con}}\times \left|{\mathrm{T}}_{\mathrm{inside},\mathrm{air}}-{\mathrm{T}}_{\mathrm{inside},\mathrm{wall}}\right|+\mathrm{\epsilon }\times \mathrm{\sigma }\times \left|{\mathrm{T}}_{\mathrm{inside},\mathrm{wall}}^4-{\mathrm{T}}_{\mathrm{inside},\mathrm{reflected}}^4\right|}$$

(4)

Within the resulting 3D spatio-thermal mesh models, T_{inside, wall} can be queried from each 3D vertex. Here, different from prior works, this study focuses on measuring actual thermal resistance of building assemblies while taking possible static occlusions into account as discussed in the previous section.

Detecting building condensation problems in 3D

Building condensation is due to the fact that a given volume of air at a given temperature can only accommodate a certain amount of water. This is typically characterized by the relative humidity and the maximum absolute humidity. The absolute humidity is a measure of the amount of the water content in the air, which itself is a function of the air temperature. The relative humidity is a measure of the percentage of the actual water content in the air with respect to the maximum absolute humidity (i.e., the amount of maximum possible water content at a given air temperature). When the air contains more water content than the maximum possible amount at a given air temperature, the amount of the extra water vapor will condense on the surrounding surfaces. Hence, the dew point temperature (T _D ) is a function of the relative humidity (f) and indoor temperature (T_{inside, air}) that can be described by the following equation (Wanielista et al. [1997]):

$${\mathrm{T}}_{\mathrm{D}}={\left(\frac{\mathrm{f}}{100}\right)}^{1/8}\left(112+0.9{\mathrm{T}}_{\mathrm{inside},\mathrm{air}}\right)+0.1{\mathrm{T}}_{\mathrm{inside},\mathrm{air}}-112$$

(5)

The indoor temperature as well as the relative humidity are measured with the use of a thermo-hygrometer as part of the thermographic inspection. In this work, the building surface temperature (T_{inside, wall}) can be queried at the level of 3D vertices from the generated 3D spatio-thermal mesh models. Thus, the building surface areas that are likely to suffer from condensation problems are directly detected by comparing T_{inside, wall} and T _D at the level of 3D points. Here, the building surface areas relevant to condensation issues can be calculated from the mesh model. For each triangle face consisting of the three thermal points that their thermal values are below T _D , the 3D coordinates of the vertices are queried, and then the area is calculated by using the cross product of two corresponding vectors that form the triangle face.

Considering static occlusions in 3D visualization of the as-is building conditions

For minimizing the impact of static occlusions when visualizing performance metrics and thermal problems obtained from the analysis of the 3D thermal model, non-relevant objects is segmented from the reconstructed 3D scene. The idea is that if the nearest thermal point in the 3D spatio-thermal model is too far from the vertices in the geometrical baseline mesh, it is most likely that the thermal points belong to objects around the wall but not the building geometry itself. Hence, a threshold is set based on the point cloud-to-mesh registration error. To do that, the registration accuracy of the 3D spatio-thermal point cloud with the geometrical baseline mesh in the site coordinate system is examined. Mapping thermal point clouds to a geometrical baseline mesh involves transforming the 3D spatio-thermal model into the real-world site coordinate system. To determine this transformation, the closed-form solution using unit quaternions is used (Horn [1987]). The transformation from the 3D spatio-thermal model into the real-world has 7 degrees of freedom (1 for scaling +3 for translation +3 for rotation), and thus requires a minimum of three matching control points between the 3D spatio-thermal model and the real-world coordinate system Ham and Golparvar-Fard ([2013a]). These control points, $P_{\mathit{Thermal}}^i$ and $P_{\mathit{Site}}^i$, can be extracted from the known dimensions in the building environments. This is solved by minimizing the sum of squared residual errors of the Euclidean transformation for (n) control points using the following:

$${\displaystyle {\sum }_1^n\parallel e^i{\parallel }^2=}{\displaystyle {\sum }_1^n\parallel p_{\mathit{Site}}^i-sR\left(P_{\mathit{Thermal}}^i\right)-T{\parallel }^2}$$

(6)

Where, s, T, and R$\left(P_{\mathit{Thermal}}^i\right)$ indicate the scaling factor, translational offset, and rotation of 3D spatio-thermal models respectively. The resulting registration error (e) is then used as a threshold for detecting thermal points that are likely to represent static occlusions. Among all T_{inside, wall} that can be queried from each 3D vertices within the 3D spatio-thermal mesh, those thermal points belonging to non-relevant objects (i.e., above the threshold (e)) are removed as outliers and are not considered in the calculation of the thermal resistance of building assemblies and detecting condensation problems. Once the relevant-to-building geometry thermal points are queried from 3D spatio-thermal mesh models, the actual R-values of the building assemblies under inspection are calculated by using Equation (4), and the building areas associated with condensation issues are detected by using Equation (5).

Then, for 3D visualization, the measured R-values and the comparison of T_{inside, wall} and T _D at the level of 3D points are converted into the corresponding RGB color value. To visualize the measured metrics and problems, the visualization platform provides several navigation modes such as a 3D free-flight navigation to efficiently visualize performance metrics and potential issues with the related building views. Similar to most engineering 3D viewers, when a camera is visited in the reconstructed 3D scene, the user can pan, tilt, and zoom-in & out for exploring the particular building areas in a mesh. For integrated visualization of the distribution of the actual R-value, the detected building areas with condensation issues, and the related geometrical and thermal characteristics, the proposed visualization platform leverages the following data structures:

• (1)
  $$P_1^e,P_2^e,\dots ,P_n^e$$

  : a set of relevant-to-building geometry 3D mesh vertices where $P_i^e=X_i^e,Y_i^e,Z_i^e,RG{B}_i^{R-\mathit{value}},RG{B}_i^{\mathit{Condensation}},RG{B}_i^{\mathit{SurfaceTemperature}}$ encapsulates the 3D location $\left(X_i^e,Y_i^e,Z_i^e\right)$, the color value $\left(RG{B}_i^{R-\mathit{value}}\right)$ of each vertex which corresponds to the measured R-value according to the given color spectrum, the color value $\left(RG{B}_i^{\mathit{Condensation}}\right)$ of each vertex which is a metaphor based on traffic light colors to represent problematic and non-problematic areas using red and green colors respectively, and finally the color value $\left(RG{B}_i^{\mathit{SurfaceTemperature}}\right)$ of each vertex associated with a temperature reading averaged from all thermal images that have observed the vertex

• (2)
  $$P_1^g,P_2^g,\dots ,P_m^g$$

  : a set of 3D points where $P_i^g=X_i^g,Y_i^g,Z_i^g,RG{B}_i^{\mathit{Geometry}}$ encapsulates the 3D location $\left(X_i^g,Y_i^g,Z_i^g\right)$ and the color $\left(RG{B}_i^{\mathit{Geometry}}\right)$ of each point in 3D building geometrical point cloud which is the average from the colors in all digital images that observed the point

• (3)
  $$C_1^T,C_2^T,\dots ,C_l^T$$

  : a set of thermal cameras representing the thermal camera projection matrix based on the relative pose with respect to the built-in digital camera

• (4)
  $$C_1^D,C_2^D,\dots ,C_l^D$$

  : a set of built-in digital cameras representing the digital camera projection matrix for each camera registered in a reconstructed 3D scene

• (5) A set of vertices that form each face

• (6) The number of vertices and faces in a mesh

• (7) The number of feature points and cameras that observe the points

• (8) Mapping relationship between points and cameras that observe the points

Results and discussions

The first five rows in Table 2 present the 3D thermal modeling metrics and the results from the case studies. In both cases, the computational time is benchmarked on a regular engineering workstation (an Intel i7 960 core with 24GBs RAM and NVIDIA GeForce GTX 400), and reasonably detailed 3D mesh models are generated in each case. Table 2 also shows the environmental measurements that were conducted during the experiments. Here, rather than concluding a single value of the actual thermal resistance for a building assembly in a deterministic way, Table 2 presents the range that the R-value for inspected building assemblies is most likely to be included.

Table 2 Experimental results of 3D thermal modeling and environmental measurements

Figures 4, 5, 6, and 7 illustrate the distribution of the surface temperatures and the measured R-values at 3D points for wall assemblies and windows in case #1 and #2 respectively. Compared to prior works (Albatici and Tonelli [2010]; Dall'O' et al. [2013]; Fokaides and Kalogirou [2011]; Madding [2008]) which leverage a single measurement of actual thermal resistance for each building assembly using a single or a few temperature data, the proposed method can provide more detailed representation of the heat transfer conditions for building assemblies at the level of 3D points. Figures 8 and 9 show examples of visualizing the distribution of the R-values along with the corresponding building geometry in 3D. The 3D visualization of thermal resistances using different color gradients can provide an opportunity for building practitioners to easily and quickly understand the distribution of the actual thermal resistance in building environments. Because static occlusions are taken into account, the proposed method has potential for improving the accuracy of measurements. In both cases, by using the equation (6), the registration errors between 3D thermal point cloud and the geometrical baseline mesh are within a reasonable practical range (Table 2). Based on the calculated registration errors, it is observed that 8.8% (case #1) and 43.8% (case #2) of the thermal points were related to non-relevant objects around wall (e.g., Figures 8 and 9(b) and (c)). Thus, those thermal points are not considered when calculating the thermal resistances in 3D. Although it is attempted to minimize the impact of such static occlusions, false negatives are still possible due to incomplete segmentation. Examples are shown as oval-shaped areas in the Figures 8 and 9 (e), which causes inaccurate thermographic inspection for those areas. Here, the proposed method can also be used for the purpose of commissioning new buildings in addition to retrofit purposes during the operational phase. Unlike occupied buildings, there is minimal occlusion during commissioning inspection. Here, the proposed method can inspect if the new building is operating as expected by the design.

Figure 4

Point-level temperature and actual R-values for the wood frame walls in case study #1.

Figure 5

Point-level temperature and actual R-values for the single-glazed clear window in case study #1.

Figure 6

Point-level temperature and the actual R-values for concrete masonry walls in case study #2.

Figure 7

Point-level temperature and the actual R-values for single-glazed clear windows in case study #2.

Figure 8

3D visualization along with the building geometry for case #1. The color spectrums shows variation between 0.14 to 1.08 m2K/W. (a): 3D reconstructed built environments, (b) and (c): thermal resistances of exterior walls with True Positive detection of occlusion, (d): 3D thermal distribution with True Positive detection of occlusion, (e): thermal resistances of exterior walls with False Negative detection of occlusion, and (f): thermal resistances of a window component.

Figure 9

3D visualization along with the building geometry for case #2. The color spectrums shows variation between 0.15 to 1.25 m2K/W. (a): 3D reconstructed built environments, (b) and (c): thermal resistances of exterior walls with True Positive detection of occlusion, (d): 3D thermal distribution with True Positive detection of occlusion, (e): thermal resistances of exterior walls with False Negative detection of occlusion, and (f): thermal resistances of a window component.

Beyond providing surface temperature data, the proposed method can also improve the quality of interpreting building thermography by detecting and localizing moisture problems on building envelopes and visualizing the outcome in 3D along with the corresponding geometrical characteristics. Figures 4, 5, 6, and 7 illustrate the dew point temperature (dotted line) and the distribution of surface temperatures for both case studies. These figures demonstrate noticeable deviations between these two temperatures and further highlight the prominence of the point-based measurements. Compared to the built-in option available in current thermal cameras (Hoff [2011]) for detecting condensation problems, the proposed method has potential to improve the current practices by estimating the size of the building areas associated with condensation issues. In Figure 10, building surface areas that are likely to suffer from condensation issues are visualized in 3D ((a) in Case #1 ≅0.0046m², (b) and (c) in Case #2 ≅0.0159m²). These results are consistent with inspection observations conducted at the time of data collection. When thermal images were captured, the surface temperatures of these areas were lower than the dew point. As shown in both cases, the condensation issues typically starts from around windowsills. To minimize the impact of these detected condensation problems, homeowners can (1) reduce the dew point temperature by decreasing the relative humidity or the indoor air temperature; (2) replace or add new insulation layers to increase the surface temperature on the areas where insulation layers have been missed or degraded; and finally (3) adjust the ventilation fan or better benefit from natural ventilation to increase the wind velocity around the problematic areas.

Figure 10

3D visualization of the building areas identified with possible condensation issues (the color-coding is based on the metaphor of traffic light colors) (a): case #1, (b) and (c): case #2.

The timing of an inspection would impact the capability of detecting condensation issues. To continuously monitor changes in the environmental conditions and study their impacts on condensation problems, one can place environmental sensors on building interior surfaces and continuously monitor the relative humidity and also the surface and inside air temperatures. However, in practice, only a few environmental sensors could be placed per element. As such, considering the variations of thermal performance over surface areas, it is difficult to ensure if the surface condition measured by a few sensors is representative of the entirety of the building assemblies. Here, the proposed method has potential to provide a solution for selecting the proper locations of installing a limited number of environmental sensors for detecting condensation problems. As a complementary step, after detecting the condensation problems from the proposed one-time thermography-based method, the environmental sensors can be placed at the detected areas for continuous monitoring. This will allow proper positioning of a limited number of sensors and improves the efficiency of monitoring.

Domain expert evaluation and discussions

To discuss the perceived benefits of the proposed method, face-to-face and phone interviews were conducted with eleven actual domain experts. The subjects had 2 to 10 years of practical experiences in energy auditing of existing buildings. They were first asked about the current best practices and bottlenecks in thermographic inspections for energy diagnostics in existing buildings. All subjects responded that today, thermography captured from existing buildings is considered the most effective tool for recording and communicating the as-is energy performance conditions to building owners. However, thermography is primarily used for qualitative documentations in support of leveraging the Home Energy Rating System (HERS) for reporting purposes at a later stage. Most responders used the word "qualitative" to describe the current process. For example, they noted that "Audits can take a long time, up to several hours for large residential houses. IR imaging is done piece-wise through a building. It takes detailed documentation to know where each image was taken and what it is supposed to be showing. This can be confusing, and is often detrimental to the client's understanding, even with a good service provider explaining discoveries.", "Sometimes it is hard to tell where the thermal image is coming from in the building. Often, our audits last about 2–4 hours. In this time, it is hard to note which image was taken where within the building."

Figure 11 illustrates the summary of the responses from the Likert questionnaires on the usefulness of each feature (1 = poor, 5 = excellent). As shown, participants generally felt that each features in the proposed work would be helpful for reducing the bottlenecks in current practices on thermography-based building condition assessment, e.g., "This is a very interesting step in the direction of recalling and identifying locations of leakage. This visualization tool will dramatically increase the potential for accurate assessments of issues.", "This visualization tool can be very efficient in detecting leaks in building insulation.", "In the summary, it would be useful to see an entire room in 3D to capture the proposed visualization tool's differentiating usefulness that cannot be done with 2D imaging."

Figure 11

Summarized responses (averaged over all subjects) from the Likert scale questionnaires about the usefulness of each feature (1 = poor, 5 = excellent).

The survey results show the potential of the proposed 3D visualization method for assessing the as-is building conditions as an intuitive communication tool. Since the main purpose of the proposed method is to provide guidance for making efficient and effective energy retrofit decisions by diagnosing and visualizing the as-is building conditions, the proposed method only needs to be conducted once per regular building diagnostic process (e.g., annual building inspection) and does not need to be performed more frequently. Hence, it does not add significant burden to the building practitioners, and allows the proposed method to be repeatable in parallel with thermographic inspection. The proposed method can inspect very small surfaces with potential performance problems (in fact, at the vertex-level of a mesh). This level of granularity, for example, would be helpful when installing blown-in loose-fill insulation is considered as one of the retrofit alternatives to improve the thermal resistance of the defected areas. However, if the detected areas are too small, retrofitting such problems is not likely to be practically feasible due to technical challenges and long payback period. Thus, practical retrofit decisions need to be taken up to a certain minimum size of retroffitable surface. The proposed method chooses not to dictate the minimum size of the retrofittable building area. Rather, by presenting the measurements at the point-level, the proposed method lets the building practitioners set their desired value prior to assessing different energy efficiency retrofit alternatives.

Despite such perceived and potential benefits, several participants noted that there is still room for improvement to alleviate bottlenecks in current practices, which are remaining open:

• 1) Difficulty in in-situ measurement of as-is thermal properties for building assemblies: Few participants had concerns about the difficulty in accurate measurement of thermal properties in building environments. Here, following prior works (Albatici and Tonelli [2010]; Dall'O' et al. [2013]; Fokaides and Kalogirou [2011]; Madding [2008]), the deviation between the measurements and the notional values was examined. It is noted that wall assembles are not the best choice for such experiments since wall assemblies typically have multiple layers and are likely to suffer from non-uniform and time-variant degradations. Hence, in this case, using the notional R-values as the ground truth for the as-is thermal resistances may not be accurate. In contrary to walls, since windows are mostly standardized and their thermal properties are more likely to be consistent over time, their actual thermal resistance is less likely to deviate from the notional values derived from the as-built documents. Thus, window components were selected for experiments. It is not a trivial task to accurately estimate the thermal properties of the old building assemblies such as the case studies of this paper. Here, the estimation is based on (ColoradoENERGY [2013]; The Engineering Toolbox [2013]) which provide recommendations for such R-values and (European Standard (EN) [1997]). As a result for single-glazed clear windows, the averaged deviations were 12.81% (= (0.301-0.260)/0.301) for case #1 and 10.37% (= (0.301-0.274)/0.301) for case #2. Following the environmental assumptions presented in (Albatici and Tonelli [2010]; Dall'O' et al. [2013]; Fokaides and Kalogirou [2011]; Madding [2008]), the findings in the experiments are consistent with what was reported in those literatures (Section Thermal resistance (R-value) of building assemblies). Here, systematic errors on measuring the input variables for Equation (5) need to be considered. If the thermographic inspection is conducted in outdoor environments, the measurement accuracy would be affected by various external factors such as wind, sun radiation, and shadows. Since the temperatures are measured from inside surfaces and crumpled aluminum foils located in indoor environments, the accuracy of measuring those input variables is typically the same as it is claimed in the technical specification of the thermal camera. As can be seen in the Table 1, the measurement accuracy of the consumer-level thermal camera used is 2°C or ±2% of reading. Here, another prevalent challenge for accurate measurement in thermal resistances in a non-destructive manner is that the measurement needs to be conducted under steady-state conditions as discussed in Section (Environmental Assumptions and Experimental Setups). Despite significant efforts to form a steady-state heat transfer condition during thermographic inspection (e.g., the timing of data collection and experimental setups prior to image data collection as described in Section(Environmental Assumptions and Experimental Setups), in practice, it is not trivial to maintain a perfect steady-state condition during thermogaphic inspection. More research needs to be conducted on how a steady state condition of heat transfer can be continued during thermographic inspection and how the measurement errors caused by non-steady-state conditions can be accounted for and minimized for more accurate analysis;

• 2) Continuous measurement to form time-series of thermal resistances: Besides the measurement accuracy issues, a few responders described `4D visualization' as a desired feature, e.g. "Maybe a 4D visualization done with the presence of all the objects that exist in the building.", "A time series IR webcam of a building over the course of a year." In prior work on measuring actual U-values (Fokaides and Kalogirou [2011]), the results were validated based on different sources of information: instantaneous surface temperature data samples of 10 measurements obtained from a thermal camera and continuous time-series heat flux data collected from HFMs over a period of 168 hours. According to the experimental results, it is shown that the outcomes of the two methods were very close to each other in terms of their accuracies. However, if surface temperature data can be collected continuously, the actual heat resistances can be modeled and explored in 4D (3D + time) as opposed to static information. Visualizing dynamic energy performance metrics in form of a 3D animation can significantly improve the understanding of time-varying heat transfer phenomena. This also enables to provide an opportunity to analyze the impacts of building envelope retrofit decisions. For example, energy auditors can conduct a before/after analysis on thermal resistances when additional insulation layers are being installed. Such analyses would support implementation of periodic building maintenance programs. More research on 4D visualization of energy performance metrics need to be conducted; and finally

• 3) Minimizing false negative detection for static occlusions: One survey participant noted that "Accounting for furniture and other obstructions seems like a challenging task". The proposed method of setting a geometrical distance threshold has potential to ignore those building areas with static occlusions. Nevertheless, as discussed in the previous section, false negatives can still occur due to incomplete segmentations. In the experiments, those happen around thin objects on walls such as frames (e.g., Figures 8 and 9(e)) or tiny objects around windows (Figure 10(b) and (c)). This is because the distance between thermal points representing such objects in 3D point clouds and the nearest vertex in a geometrical baseline mesh is most likely to be within the threshold based on the registration error. As a result, the thermal resistance of the occluded building assemblies was not accurately calculated. More research needs to be conducted to better segment non-relevant objects from the reconstructed 3D scenes.

Conclusions

Effective building diagnostic tools should represent abnormal building conditions in an intuitive format to allow energy auditors better communicate the as-is building conditions to the homeowners. Toward this goal, this paper presents a method for visualizing the as-is building condition assessment at the level of 3D points. The outcome of the analyses are visualized within a 3D environment, and static occlusions are accounted for. In this 3D environment, practitioners can navigate and interactively query information about the building geometry, surface temperature data, thermal resistances, and analyze condensation problems. The proposed method was tested in a residential building and an instructional facility, and the following domain expert evaluation was conducted. By converting surface temperature data sensed from a consumer-level single thermal camera into 3D visualization of the actual R-value distributions and the location information of possible condensation problems, building practitioners can better understand the as-is building conditions. Future works include addressing the following challenges in (1) forming and maintaining a steady state condition of heat transfer; (2) studying 4D visualization of energy performance metrics based on time-series of building surface temperatures; (3) minimizing false negative segmentations for static occlusions; and (4) Varying the level of visualization for both global and local inspection of as-is energy performance based on different Levels of Detail (LOD). These are currently being explored as part of ongoing research.