報(bào)告題目：Consistency of the oblique decision tree and its boosting and random forest

報(bào)告人：詹浩然

邀請(qǐng)人：黃磊

報(bào)告時(shí)間：2025年03月14日16:00-17:00

報(bào)告地點(diǎn)：kaiyun開(kāi)云官方網(wǎng)站犀浦校區(qū)3教X30423

摘要：Classification and Regression Tree (CART), Random Forest (RF) and Gradient Boosting Tree (GBT) are probably the most popular set of statistical learning methods. However, their statistical consistency can only be proved under very restrictive assumptions on the underlying regression function. As an extension to standard CART, the oblique decision tree (ODT), which uses linear combinations of predictors as partitioning variables, has received much attention. ODT tends to perform numerically better than CART and requires fewer partitions. In this paper, we show that ODT is consistent for very general regression functions as long as they are L^2 integrable. Then, we prove the consistency of the ODT-based random forest (ODRF), whether fully grown or not. Finally, we propose an ensemble of GBT for regression by borrowing the technique of orthogonal matching pursuit and study its consistency under very mild conditions on the tree structure. After refining existing computer packages according to the established theory, extensive experiments on real data sets show that both our ensemble boosting trees and ODRF have noticeable overall improvements over RF and other forests.

報(bào)告人介紹：詹浩然，2023年博士畢業(yè)于新加坡國(guó)立大學(xué)，目前在新加坡國(guó)立大學(xué)統(tǒng)計(jì)與數(shù)據(jù)科學(xué)系做Research Fellow，科研興趣主要是統(tǒng)計(jì)學(xué)習(xí)及其理論，包括神經(jīng)網(wǎng)絡(luò)、隨機(jī)森林、boosting方法、大模型的統(tǒng)計(jì)理論。研究成果刊登于學(xué)術(shù)期刊The Annals of Statistics, Bernoulli, Computational Statistics & Data Analysis。