Open Access is an initiative that aims to make scientific research freely available to all. To date our community has made over 100 million downloads. It’s based on principles of collaboration, unobstructed discovery, and, most importantly, scientific progression. As PhD students, we found it difficult to access the research we needed, so we decided to create a new Open Access publisher that levels the playing field for scientists across the world. How? By making research easy to access, and puts the academic needs of the researchers before the business interests of publishers.
We are a community of more than 103,000 authors and editors from 3,291 institutions spanning 160 countries, including Nobel Prize winners and some of the world’s most-cited researchers. Publishing on IntechOpen allows authors to earn citations and find new collaborators, meaning more people see your work not only from your own field of study, but from other related fields too.
Accurate real-time prediction of diesel soot emissions is essential for meeting stringent emission regulations and optimizing aftertreatment performance. This study presents a data-driven framework for engine-out soot mass prediction, developed using experiments on an SC7H turbocharged diesel engine under the non-road transient cycle. A comprehensive dataset of soot concentration and engine sensor variables was analyzed using correlation-based network analysis, which identified diesel particulate filter (DPF) inlet oxygen concentration as a key factor in soot formation and oxidation. Two predictive models, a Light Gradient-Boosting Machine and a long short-term memory (LSTM) neural network, were trained to capture transient soot dynamics. A self-learning fusion strategy integrated their outputs into a meta-learner, forming an ensemble model. The fused model improved prediction accuracy by 7–10% over individual models and effectively tracked rapid emission fluctuations. This virtual sensing approach enables precise soot loading estimation, supporting optimal DPF regeneration timing and advanced emission control strategies, and offers a scalable methodology for environmentally sustainable diesel engine design in Industry 4.0.
*Address all correspondence to: wenkai.chen@uni-hamburg.de
1. Introduction
Diesel engine particulate emissions, particularly soot or black carbon, pose significant environmental, regulatory, and health challenges. Although heavy-duty diesel vehicles comprise a small fraction of the global vehicle fleet, they are responsible for over 80% of NOx emissions and more than 90% of particulate emissions from the transportation sector [1]. These emissions not only degrade air quality but also contribute disproportionately to public health burdens, especially for communities living near major freight routes [2].
Stringent regulatory actions have been introduced globally to curb these harmful pollutants. The European Union’s Euro VI standard, enforced since 2013, has slashed permissible NOx and PM emissions by up to 95% and 97%, respectively [3, 4]. Similarly, the U.S. Environmental Protection Agency (EPA) adopted the “Clean Trucks” final rule in December 2022, setting tighter tailpipe limits for particulate matter and ozone precursors beginning in model year 2027 [5]. These evolving regulations underscore the urgent need for robust tools capable of managing and predicting diesel particulate emissions in real-world driving conditions.
Accurate, real-time prediction of soot output is critical for designing efficient engines and optimizing aftertreatment systems such as diesel particulate filters (DPFs). To address this challenge, numerous physics-based models have been proposed. Bissett [6] developed one of the earliest mathematical models for the thermal regeneration of wall-flow monolith DPFs, laying the foundation for understanding soot oxidation under controlled heating. Building on such foundations, Li et al. [7] proposed a soot loading estimation model for heavy-duty engine DPFs that also incorporated passive regeneration characteristics, enabling more accurate soot mass tracking under real driving cycles. Wang et al. [8] advanced this direction by introducing a multilayer equivalent permeability model based on collection mechanisms, which effectively captures both deep-bed and cake-filtration regimes while maintaining computational efficiency for control-oriented applications. Similarly, Zhang et al. [9] employed a multidisciplinary design optimization approach for DPFs in composite regeneration processes, simultaneously considering filtration performance, regeneration efficiency, and backpressure constraints. Huang et al. [10] investigated a model-based estimation method for soot loading in catalyzed DPFs, demonstrating improved accuracy by leveraging catalyst activity and reaction kinetics. These works highlight that, while physics-based models can provide interpretability and capture filtration mechanisms, their performance under highly transient engine conditions is often limited by parameter sensitivity and calibration requirements. These traditional methods often rely on simplified empirical maps using engine speed and torque derived from steady-state testing. While effective in controlled environments, these approaches fail to capture the complex, nonlinear dynamics of soot generation under transient conditions, where factors such as fuel injection strategy, air–fuel ratio, turbocharger response, and exhaust gas recirculation (EGR) modulation play significant roles.
Recent advances in data science have led to a growing adoption of machine learning (ML) algorithms for engine emission modeling [11]. Compared to physics-based approaches, which require detailed chemical and physical submodels and often become computationally prohibitive for transient operating conditions, ML models can efficiently capture complex, nonlinear relationships among high-dimensional engine parameters and are fast to evaluate in real time. For example, Shahpouri et al. [12] demonstrated that gradient-boosting and deep learning architectures could successfully model soot emissions in a compression ignition engine, achieving high accuracy without explicit in-cylinder combustion modeling. Extending this, Shahpouri et al. [13] proposed hybrid ML frameworks combining physical insights with systematic model selection procedures, which further improved soot prediction robustness under diverse operating conditions. More recently, Mohammad et al. [14] explored a physics-oriented and ML-based comodeling strategy, using dimensionality reduction techniques to identify dominant predictors before applying regression algorithms, thus balancing interpretability and predictive power. However, most of these efforts cannot solve the problem of predicting engine-out soot emissions from a diesel engine under the non-road transient cycle (NRTC) conditions. To address this gap, our work develops a data-driven soot loading prediction framework based on measurements from the NRTC, which closely replicates real-world driving conditions for off-road diesel engines.
Our approach begins with the collection of a comprehensive dataset from a fully instrumented engine test bed, capturing time-resolved signals from multiple subsystems, including engine torque and speed, fuel injection parameters, temperature sensors, air intake sensors, and auxiliary equipment, along with the accumulated soot mass at the DPF inlet. Prior to model training, we perform correlation-based data analysis to identify key predictive factors and quantify their interrelationships across multiple NRTC runs. This step provides an interpretable variable selection mechanism, ensuring that subsequent ML models focus on the most influential parameters. Two complementary predictive models are then developed: a Light Gradient-Boosting Machine (LightGBM) to capture nonlinear feature interactions and a long short-term memory (LSTM) network to learn temporal dependencies in the emission dynamics. After identifying the key contributing factors, a second network analysis is conducted to visualize their interconnections, offering physical insight into how engine operating conditions collectively influence soot accumulation.
This integrated methodology not only advances the theoretical understanding of DPF soot-loading mechanisms but also aligns with the principles of Industry 4.0 [15], where interconnected sensing [16], intelligent analytics, and real-time decision-making [17] form the backbone of advanced manufacturing and powertrain systems. By leveraging high-resolution sensor data from NRTC conditions, our approach can be embedded into IoT-enabled control architectures or serve as the predictive core of digital twins for diesel engine aftertreatment systems. Such integration enables closed-loop optimization of combustion and filtration processes, improving both performance and environmental compliance.
2. Methodology
2.1 Experimental setup
All experiments were carried out on a turbocharged, intercooled diesel engine, which is a 7.2-L, inline 6-cylinder, high-pressure, common-rail engine. The engine was mounted on an eddy-current dynamometer in a test cell. An array of sensors and instrumentation was used to monitor key engine and emission parameters. These included intake airflow, intake manifold pressure and temperature, fuel flow rate, rail pressure, engine speed, torque, coolant temperature, and exhaust conditions. In-cylinder pressure was recorded with a piezoelectric transducer for combustion analysis, and an AVL 483 optical sonic soot sensor was installed at the DPF inlet to measure real-time soot concentration. The AVL 483 sensor operates by emitting modulated infrared light across the exhaust stream and detecting the acoustic pressure waves generated as soot particles absorb the light and thermally expand. The amplitude of the sensor’s signal is directly proportional to soot mass concentration in the exhaust, allowing continuous measurement of soot mass. Figure 1 illustrates detailed information about the experimental setup and specific measuring mechanisms of the AVL soot sensor. Moreover, the detailed specification parameters of the diesel engine setup are summarized in Table 1.
Figure 1.
Experimental setup for soot mass measurement. (a) Airflow system comprising the engine, intake and exhaust paths, EGR system, DPF, AVL soot sensor, and network analysis unit. (b) Soot detection principle of the AVL sensor.
Project
Specification
Engine type
Six-cylinder in-line/inter-cooling
Intake method
Turbocharged and intercooled
Rated power/kW
192
Rated speed/r⋅min−1
2,200
Max torque/N⋅m
1,000
Speed at max torque/r⋅min−1
1,300–1,700
Number of valves per cylinder
4
Table 1.
Technical specifications of the SC7H260G4 diesel engine.
2.2 Data collection under NRTC
To ensure the engine experienced a wide range of operating conditions, we used the NRTC as the test schedule. The NRTC is a transient test cycle developed by the U.S. EPA and EU for off-road diesel engines. It lasts 1,238s, and involves rapid changes in engine speed and torque every second to mimic real-world operations such as accelerations, decelerations, and idle periods. Figure 2 illustrates the normalized speed and torque profile over time in the NRTC, with values given as fractions of the engine’s maximum speed and torque. This cycle was chosen because it closely represents actual in-use duty cycles of nonroad machinery and is even more transient than on-road driving cycles, thereby providing a stringent test for the prediction model.
Figure 2.
Reference speed and reference torque of the engine in the NRTC.
We conducted over 100 repeated NRTC tests on the engine under consistent conditions to build a robust dataset for model training. In total, the dataset comprises 153,106 time-step samples of measured parameters and the corresponding soot concentration. Each sample corresponds to a 1-second interval of the cycle. There are 11 input features, which serve as predictors in our model, and one output target (soot mass concentration), as shown in Table 2. The input features mainly consist of engine speed (NE), engine torque (NT), main fuel injection quantity (QFIN), rail pressure (PFIN), main injection timing (TFIN), boost pressure (PIM) and temperature (THA), coolant temperature (THW), EGR valve position (LEGR), and oxygen concentration at the DPF inlet (CEXREAL). The output label is the real-time soot mass concentration at the DPF inlet. All sensors were sampled at 1 Hz to match the cycle’s resolution. This comprehensive set of variables allows the model to account for fuel supply, air management, combustion timing, and aftertreatment inlet conditions, all of which are expected to influence soot formation and oxidation.
Type
Variables
Variables abbreviation
Input variables
Engine speed
NE
Engine torque
NT
Main injection quantity
QFIN
Injection pressure
PFIN
Main injection timing
TFIN
Intake manifold pressure
PIM
Intake air mass flow rate
AIR_FLOW
Intake air temperature
THA
Coolant water temperature
THW
EGR valve opening
LEGR
Measured O2 concentration
CEXREAL
Output variables
Real-time soot mass
Soot
Table 2.
Details of the input and output variables collected during the NRTC.
2.3 Model construction
The overall model construction process begins with acquiring a soot mass emission dataset from multiple onboard sensor sources, covering a wide range of engine operating parameters, intake and exhaust conditions, and aftertreatment measurements. The raw measurements undergo a comprehensive data preprocessing stage to address missing values, remove outliers, and correct long-tail distributions, ensuring both accuracy and statistical balance. The cleaned dataset is then transformed through augmentation to expand the diversity of training examples and normalization to standardize variable ranges, which improves model stability and comparability between features. After transformation, the dataset is split into training and testing subsets in an 8:2 ratio for predictive model development.
Two complementary algorithms are deployed: the LightGBM and the LSTM recurrent neural network (RNN). LightGBM uses a histogram-based decision tree growth method in which continuous features are discretized into a fixed number of bins, and feature statistics are accumulated in histograms across the training data. This approach greatly reduces memory consumption and computational cost while maintaining scalability for medium-to-large datasets. The LSTM network is designed to capture temporal dependencies in soot emissions by employing memory cells and gating mechanisms – input, forget, and output gates – that store relevant historical information, filter out less useful past states, and update predictions dynamically.
Once both models are initially trained, hyperparameters are fine-tuned to accelerate convergence and reduce overfitting. During testing, batch normalization is applied to ensure consistent data formats with those used in training. Finally, a self-learning ensemble strategy is applied: The predictions from the LightGBM and LSTM models are concatenated to form new feature vectors, which are combined with the original soot dataset and passed to a logistic regression meta-learner. As shown in Figure 3, this meta-learner automatically determines the optimal weighting of each submodel’s predictions, producing a fused output that improves both accuracy and generalization by exploiting the complementary strengths of the two base models.
Figure 3.
Workflow of the self-learning ensemble framework. Predictions from LightGBM and LSTM models are combined into new features and processed by a stacking meta-learner for soot mass estimation.
For model optimization, the mean squared error (MSE) is used as the loss function during training. Model performance is quantitatively evaluated using three accuracy-related metrics: mean absolute error (MAE), MSE, and root mean squared error (RMSE), defined as follows:
MAE=1N∑i=1N|yi−y^i|,E1
MSE=1N∑i=1N(yi−y^i)2,E2
RMSE=1N∑i=1N(yi−y^i)2,E3
where yi denotes the measured soot mass concentration for the i-th operating point, y^i denotes the corresponding predicted value, and N is the total number of operating points.
3. Data analysis
3.1 Variables analysis
The torque–speed profile recorded during the NRTC shows that the experimental diesel engine operated within a torque range of 0–1000 N⋅m and a speed range of 600–2400 rpm, covering the full spectrum from idle to maximum speed. Most operating points were concentrated above 1200 rpm, reflecting that the NRTC protocol effectively reproduces real-world load conditions. As shown in Figure 4, when torque and speed variations are viewed alongside the corresponding soot mass concentration trends, it becomes evident that soot emissions do not follow a steady pattern but instead exhibit frequent sharp peaks. These spikes are closely linked to abrupt transitions in engine operating conditions.
Figure 4.
NRTC torque and speed profiles over time, showing rapid load fluctuations and transient operating conditions. These changes induce sharp transient soot peaks, emphasizing the need for time-aware prediction models.
As shown in Figure 5, the first figure shows the real-time curve of soot emission detected from the AVL 483 sensor during a single NRTC. It can be seen that diesel engine soot emissions do not fluctuate regularly within a fixed range. Due to the presence of operating conditions in the NRTC test cycle where engine speed and torque undergo large, instantaneous jumps, soot emissions often exhibit sudden peaks. Moreover, we adopt extra three figures to illustrate the variation relationship between soot mass concentration and torque at different time periods within the cycle, which can be classified into three operating conditions from idle start, medium-speed operation, and acceleration. In the idle start phase (0–200 s), torque and speed both rise gradually from low-load conditions. Here, low air–fuel ratio and low in-cylinder temperature create extensive fuel-rich zones, limiting oxidation and promoting particle agglomeration, which drives a steady increase in soot concentration as torque builds. During the medium-speed phase (200–400 s), torque and speed fluctuate sharply, and soot concentration remains relatively insensitive to torque magnitude. This is because both rapid load increases and sudden load reductions can raise particle formation, high-load low-speed conditions encourage fuel cracking and incomplete oxidation, while load-shedding events disrupt combustion stability. In the acceleration phase (600–800 s), the overall trend is a decrease in soot concentration with increasing torque, as higher engine speeds enhance air–fuel mixing and oxidation of soot formed earlier in low-speed, high-load conditions. However, as can be seen from the last figure in Figure 5, a sudden torque drop around 700 N⋅m can worsen combustion conditions, causing a temporary surge in soot emissions.
Figure 5.
Soot detected from the AVL 483 sensor versus time, and soot concentration versus torque across three NRTC phases (0–200 s, 200–400 s, and 400–800 s).
To further investigate the emission characteristics of the diesel engine at a stable rotational speed, it can be seen from the engine speed variation chart in Figure 4 that during the period from 850 to 1000 s, the engine speed remains essentially stable around 1950 rpm, despite a sudden drop from 1950 rpm to 1550 rpm occurring in the middle of this interval. Figure 6 presents the relationship between soot mass concentration and engine speed within this 850–1000 s segment of the NRTC. The figure shows that when the engine speed is 1950 rpm, the transient soot emission is mainly distributed between 0.75 and 1.75 mg/m3; when the speed is 1550 rpm, the transient soot emission is mainly distributed between 1.5 and 1.75 mg/m3. This directly indicates that when the engine speed suddenly decreases, incomplete combustion of diesel fuel and soot particles occurs, leading to a noticeable increase in soot emissions.
Figure 6.
Relationship between soot emission and speed during the 850–1000s period of the NRTC.
By combining the torque–speed evolution and soot–torque relationships into a single analysis, the data clearly demonstrate that transient dynamics strongly influence particulate emissions. This coupling between mechanical load variation and combustion chemistry underscores the importance of predictive models that can capture nonlinear, time-dependent interactions in diesel engine operation.
3.2 Correlational analysis
With the processed dataset, we conducted a correlation analysis among all sensor features and the soot output. Figure 7 shows the Pearson correlation heatmap for the 11 input features and soot concentration. This analysis provides an initial understanding of how each variable individually relates to soot emissions. The most influential parameter was found to be the oxygen concentration at the DPF inlet (CEXREAL), which exhibited a correlation coefficient of about − 0.61 with soot mass. This indicates a strong negative correlation: Higher oxygen concentration in the exhaust (upstream of the DPF) is associated with lower soot mass, which makes physical sense. Oxygen helps oxidize soot particles; when more O2 is present, soot can burn more completely before exiting to the DPF, thus measured soot concentration drops. The data indeed confirmed that at operating points with abundant excess air, soot levels were suppressed, consistent with known combustion behavior and our correlation finding. Other variables showed moderate correlations: For instance, engine torque and fuel injection quantity had positive correlations with soot, where heavier load and more fuel generally produce more soot. While boost pressure had a negative correlation, where higher boost pressure implies more air and better combustion, reducing soot. These pairwise correlations were further interpreted as a network: Each sensor variable can be considered a node, and weighted links between nodes represent the strength of correlation. In this correlation analysis, the DPF inlet O2 emerged as a central node strongly linked (negatively) to soot, whereas some other variables (e.g. coolant temperature) had relatively weak direct connections with soot. This network-centric view guided our understanding of feature importance, complementing the ML models’ internal assessments. It is noteworthy that our data-driven correlation results align well with diesel combustion theory, providing confidence that the ML models can leverage these meaningful relationships rather than spurious correlations.
Figure 7.
Pearson correlation heatmap for 11 input variables and soot concentration, indicating a strong negative correlation between DPF inlet oxygen concentration (CEXREAL) and soot.
Additionally, engine torque exhibited a moderate positive correlation with soot, reflecting that higher torque often corresponds to increased fuel injection, which can increase particulate matter when not fully combusted. This was supported by the phase-wise analysis in Section 3.1, where medium-speed high-load conditions produced notable soot surges. Speed showed mixed behavior, where high speeds generally promoted better mixing but could also correlate with transient load imbalances that briefly worsen soot emissions. To bridge data-driven insights with real-time operational explainability, we developed a practical interpretability vignette as shown in Figure 8. This visualization illustrates how a combined drop in torque and inlet O2 can predict a forthcoming soot spike, prompting a recommendation for DPF regeneration. Such visual logic chains enhance model transparency and usability for field technicians, enabling predictive maintenance strategies rather than reactive responses.
Figure 8.
Interpretability vignette showing how low O2 and torque drops precede soot spikes, guiding proactive regeneration.
4. Machine learning predictions
4.1 LightGBM Model
Unlike conventional gradient-boosting algorithms such as GBDT and XGBoost in their node-splitting and data-storage approaches, LightGBM adopts a histogram-based algorithm to construct discrete splits for data nodes [18]. The core idea is to discretize continuous floating-point features into K discrete values and construct K histograms. After traversing all the training data, cumulative statistics for each discrete value in the histogram are calculated. Therefore, when the algorithm scores and selects features for each label, it can directly identify the optimal split point from the discrete histograms. This method significantly reduces memory consumption and computational load during model training, making it well suited for medium- to large-scale datasets. We employ the LightGBM for soot–mass prediction due to its histogram-based split finding and leaf-wise tree growth, which together enable fast training and inference on large, high-rate engine datasets. After the processed dataset is divided into training and testing sets, the concept of K-fold cross-validation is applied to stratify the data. This means that the output of the prediction model is obtained by averaging the outputs of the initially specified K trained models, which helps improve prediction accuracy. The end-to-end workflow, data split, fold iteration, training, and fold-averaged evaluation, follows the pipeline we designed (as shown in Figure 9).
Figure 9.
Five-fold cross-validation process for training the LightGBM model on soot mass prediction.
When the number of training iterations in the LightGBM algorithm is set to 5,000 and the loss function is the MSE, Figure 10 shows the impact of the number of folds in cross-validation on the final model score in the workflow. As can be seen from the figure, as K increases, the cross-validation score first decreases and then increases, reaching its minimum when K=5. This is because, in the K-fold cross-validation method, as K increases, the bias decreases with the increase in error, while the variance increases with the increase in error, and the computational cost also rises accordingly. For a well-performing ML model, the variance and bias should be relatively balanced to ensure both good fitting performance and strong generalization ability. Therefore, in the model workflow, the initial number of cross-validation folds was set to 5. To avoid temporal leakage, fold splits were performed using a blocked time-series approach that prevents information from future time steps from influencing model training. This ensures the integrity of predictive performance for sequential emission data.
Figure 10.
The effect of cross-validation folds on model score.
Key hyperparameters were tuned via a small grid search informed by prior experience, and the final model used num_leaves = 120, learning_rate = 0.01, min_child_samples = 30, the regression objective (MSE loss), and an L1 regularization term λl1=0.1, which yielded stable convergence without sacrificing the ability to capture sharp transients. To assess out-of-sample performance, we collected an independent test set from five additional NRTC runs (6,475 time steps not used in training). The comparative distribution of LightGBM predictions versus experimental soot mass concentration shows strong agreement in median and interquartile ranges over the dominant operating regime (approximately 0–40 mg/m3), while also reproducing the long right tail associated with rapid torque–speed excursions (as shown in Figure 11). Residual sparse outliers at very high concentrations are consistent with short, high-load transients; overall, the fold-averaged model provides reliable virtual sensing across most NRTC conditions and serves as a strong base learner for the subsequent fusion stage.
Figure 11.
Comparison of LightGBM model predictions and experimental soot mass distributions over the NRTC test dataset.
4.2 LSTM model
Long short-term memory networks are a class of RNNs specifically designed to learn long-range dependencies in sequential data [19]. By incorporating memory cells and gating mechanisms, LSTMs can retain and selectively update relevant information across time steps, making them highly effective for time-series prediction tasks. This capability is particularly valuable for modeling temporal patterns in engine emission data, where past behavior influences current outputs. The architecture of the LSTM-based RNN model is shown in Figure 12. The model consists of three layers, with the first layer containing 30 LSTM units, each layer using the ReLU activation function, and batch normalization applied in the third layer. Given the large volume of input data, the batch size for training was set to 1,200.
Figure 12.
Architecture of the LSTM-based soot prediction model, comprising three LSTM layers with ReLU activation, dropout, and batch normalization.
For a single-layer LSTM module, the number of LSTM units was set to 30. Given the large size of the input dataset, the batch size for each training iteration was set to 1,200. The detailed operation of each LSTM unit is as follows:
ft=σ(Wf⋅[ht−1,xt]+bf),Wf∈ℝ30×1200,bf∈ℝ30E4
it=σ(Wi⋅[ht−1,xt]+bi),Wi∈ℝ30×1200,bi∈ℝ30E5
C˜t=tanh(Wc⋅[ht−1,xt]+bc),Wc∈ℝ30×1200,bc∈ℝ30E6
Ct=ft⊙Ct−1+it⊙C˜tE7
ot=σ(Wo⋅[ht−1,xt]+bo),Wo∈ℝ30×1200,bo∈ℝ30E8
ht=ot⊙tanh(Ct)E9
Where:
ht is the LSTM output at time step t.
xt is the proposed data input at time step t.
Ct is the LSTM cell state at time step t.
[Wf,bf] are the weight matrix and bias vector for the forget gate.
[Wi,bi] are the weight matrix and bias vector for the input gate.
[Wc,bc] are the weight matrix and bias vector for the cell candidate state.
[Wo,bo] are the weight matrix and bias vector for the output gate.
Figure 13 shows the effect of different parameters on the training and validation errors of the LSTM network. Increasing the number of nodes in each LSTM layer increases the structural complexity of the network, which can potentially improve prediction accuracy. However, an overly complex structure can lead to excessive parameters, longer training times, and a greater risk of the “vanishing gradient” phenomenon. As shown in Figure 13, when the number of iterations is set to 1000, the training error decreases gradually with increasing node count. Nevertheless, with 50 nodes, the error curve does not converge by the end of training, while for 15 or 25 nodes, convergence begins around 700 iterations. To accelerate convergence without overly increasing complexity, the number of nodes in each LSTM layer in this model was set to 30.
Figure 13.
The effect of different input-node numbers in LSTM.
Following the training phase, we conducted five additional NRTC tests to collect 6,475 data samples, each sharing the same labels as the original soot mass emission dataset, for use as an independent test set. Similar to the LightGBM model, the trained LSTM model was validated on this dataset. The results, shown in Figure 14, indicate that for instantaneous emission values in the range of 0–25 mg/m3, the predicted soot mass closely matches the experimental data distribution. However, for values above 25 mg/m3, certain outliers in the experimental data are not well captured by the LSTM predictions, suggesting that the neural network model is less effective in accurately predicting extreme outlier emissions.
Figure 14.
Comparison between LSTM model predictions and experimental soot mass distribution over the NRTC test dataset.
4.3 Model fusion
As shown in Figures 11 and 14, while both LightGBM and LSTM serve as powerful predictive tools, the integration of correlation network analysis ensures interpretability. For example, identifying DPF inlet oxygen concentration as a dominant negative predictor allows engineers to diagnose filtration inefficiencies or detect sensor drift in real time. Such transparency transforms the framework from a pure predictive tool into a diagnostic and decision-support system, enabling predictive maintenance strategies and informed control parameter tuning. Moreover, the computational efficiency of LightGBM and the deployable architecture of compact LSTM networks make the framework feasible for real-time execution on embedded ECUs, supporting industrial-scale deployment.
Although the LightGBM model achieved high predictive accuracy on the current dataset, its decision tree-based architecture can experience accuracy degradation when the training set becomes very large. In contrast, the RNN model, which incorporates LSTM units to account for temporal dependencies in the soot mass emission dataset, contains a relatively large number of trainable parameters, making it prone to overfitting. To further improve predictive performance while mitigating the overfitting risk of individual models, we implemented a model fusion approach that integrates the complementary strengths of the submodels by aggregating their prediction outputs. A self-learning strategy was adopted (see Figure 3), in which the outputs of the LightGBM and LSTM models were concatenated and used as input features for a regularized linear regression meta-learner that adaptively weighted each submodel’s contribution.
As shown in Table 3, the fused model demonstrated a clear performance improvement over either single model on the independent test set containing 6,475 samples. Specifically, the fusion approach achieved the lowest MAE, MSE, and RMSE, indicating both higher accuracy and better generalization to unseen data. Furthermore, typical soot mass detection thresholds in commercial DPF systems range from 3 to 5 mg/m3 for triggering regeneration events. With the fusion model achieving a MAE of approximately 1.45 mg/m3 and a RMSE of around 2.4 mg/m3, the system can reliably inform control decisions without premature regenerations or compliance risks.
Method
MAE
MSE
RMSE
LightGBM
1.57669
7.76125
2.78590
Recurrent neural network (LSTM)
1.56219
7.19869
2.68303
Fusion model
1.44729
7.11167
2.40323
Table 3.
Comparison of real-time soot mass emission predictions between different models and experimental results. The fusion model reduces prediction error by 8% (MAE) compared to standalone LightGBM and LSTM models.
5. Conclusion
In this study, emission tests were conducted on a diesel engine under the NRTC to investigate the variation patterns of soot mass emissions with respect to torque over different time periods. By collecting soot mass emission data from numerous NRTCs, along with multiple onboard sensor measurements reflecting engine operating conditions, a comprehensive diesel engine soot mass emission dataset was constructed, comprising more than 150,000 data samples. Based on this dataset, two predictive models were developed: a LightGBM decision tree model and an LSTM-based RNN model, both designed to achieve accurate real-time prediction of soot mass emissions. Considering both predictive accuracy and model stability, a model fusion strategy was introduced to further enhance prediction performance. The fusion model combined the outputs of the LightGBM and LSTM models through a self-learning mechanism, thereby leveraging the strengths of both approaches.
Comparisons with experimental measurements demonstrated that, for various operating points within the NRTC conditions, the fusion model achieved MAE and RMSE values within 2.5 mg/m3. Beyond predictive accuracy, this framework supports industrial sustainability by enabling optimal DPF regeneration timing, thereby reducing unnecessary fuel consumption and associated CO2 emissions. By ensuring compliance with stringent standards such as Euro VI and the forthcoming U.S. EPA “Clean Trucks” regulations, it directly contributes to industrial decarbonization goals. Its adaptability also makes it suitable for engines using low-carbon fuels and hybrid configurations, aligning with future mobility trends.
References
1.YangJ, CheX, TanJ, QinX, DuanJ, LiuD, DuanY, XiangS, ShenN, ZhaiX, et al.Real-world emission characteristics and driving factors of diesel trucks: Insights from plume chasing experiments. Atmospheric Environment: X. 2025;25:100311. DOI: 10.1016/j.aeaoa.2025.100311
2.GagnonSClearing roadblocks for zero-emission medium- and heavy-duty transportation. Center for Climate and Energy Solutions. 2023. Available from: https://www.c2es.org/document/clearing-roadblocks-for-zeroemission-medium-and-heavy-dutytransportation/ [Accessed: 2025-August-10]
3.GrigoratosT, FontarasG, GiechaskielB, ZacharofN. Real world emissions performance of heavy-duty Euro VI diesel vehicles. Atmospheric Environment. 2019;201:348–359. DOI: 10.1016/j.atmosenv.2018.12.042
4.GioriaR, SelleriT, GiechaskielB, FranzettiJ, FerrareseC, MelasA, ForloniF, Suarez-BertoaR, PerujoA. Regulated and unregulated emissions from Euro VI Diesel and CNG heavy-duty vehicles. Transportation Research Part D: Transport and Environment. 2024;134:104349. DOI: 10.1016/j.trd.2024.104349
5.USEPA. Control of air pollution from new motor vehicles: Heavy-duty engine and vehicle standards and highway diesel fuel sulfur control requirements; final rule. Federal Register. 2001;66(12):5002–5193
6.BissettEJ. Mathematical model of the thermal regeneration of a wall-flow monolith diesel particulate filter. Chemical Engineering Science. 1984;39(8):1233–1244. DOI: 10.1016/0009-2509(84)85084-8
7.LiJ, ZhangH, WangY, YangX, ZhangY. Soot loading estimation model and passive regeneration characteristics of DPF system for heavy-duty engine. Energy. 2020;197:117239. DOI: 10.1016/j.energy.2020.117239
8.WangY, LiJ, ZhangH, YangX, ZhangY. Novel soot loading prediction model of diesel particulate filter based on collection mechanism and equivalent permeability. Fuel. 2020;282:118803. DOI: 10.1016/j.fuel.2020.119409
9.ZhangB, JE, GongJ, HuangW, FengQ. Multidisciplinary design optimization of the diesel particulate filter in the composite regeneration process. Applied Energy. 2016;181:14–28. DOI: 10.1016/j.apenergy.2016.08.051
10.HuangT, HuG, GuoF, ZhangZ, DengK. Investigation of a model-based approach to estimating soot loading amount in catalyzed diesel particulate filters. SAE International Journal of Engines. 2019;12(5):567–577. DOI: 10.4271/03-12-05-0036
11.KhuranaS, SaxenaS, JainS, DixitAPredictive modeling of engine emissions using machine learning: A review. Materials Today: Proceedings. 2021;38:280–284. DOI: 10.1016/j.matpr.2020.07.204
12.ShahpouriS, NorouziA, HaydukC, RezaeiR, ShahbakhtiM, KochCR. Soot emission modeling of a compression ignition engine using machine learning. IFAC-PapersOnLine. 2021;54(20):826–833. DOI: 10.1016/j.ifacol.2021.11.274
13.ShahpouriS, NorouziA, HaydukC, RezaeiR, ShahbakhtiM, KochCR. Hybrid machine learning approaches and a systematic model selection process for predicting soot emissions in compression ignition engines. Energies. 2021;14(23):7865. DOI: 10.3390/en14237865
14.MohammadA, RezaeiR, HaydukC, DelebinskiT, ShahpouriS, ShahbakhtiM. Physical-oriented and machine learning-based emission modeling in a diesel compression ignition engine: Dimensionality reduction and regression. International Journal of Engine Research. 2023;24(3):904–918. DOI: 10.1177/14680874211070736
15.SonyM, NaikS. Key ingredients for evaluating Industry 4.0 readiness for organizations: A literature review. Benchmarking: An International Journal. 2020;27(7):2213–2232. DOI: 10.1108/BIJ-09-2018-0284
16.ChenW, LiuS-C, ZhangJ. Ehoa: A benchmark for task-oriented hand-object action recognition via event vision. IEEE Transactions on Industrial Informatics. 2024;20(8):10304–10313. DOI: 10.1109/TII.2024.3393007
17.ChenW, ZengC, LiangH, SunF, ZhangJ. Multimodality driven impedance-based Sim2Real transfer learning for robotic multiple Peg-in-Hole assembly. IEEE Transactions on Cybernetics. 2023;54(5):2784–2797. DOI: 10.1109/TCYB.2023.3310505
18.KeG, MengQ, FinleyT, WangT, ChenW, MaW, YeQ, LiuT-Y. LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems. 2017;30
19.GreffK, SrivastavaRK, KoutnkJ, SteunebrinkBR, SchmidhuberJ. LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems. 2016;28(10):2222–2232. DOI: 10.1109/TNNLS.2016.2582924
Written By
Wenkai Chen
Submitted: 31 August 2025Reviewed: 16 September 2025Published: 28 November 2025
Open access peer-reviewed chapter
