The article considers the problem of estimating linear parameters in stochastic regression models with Gaussian noises, such as an autoregression of the first order, threshold autoregression, and some others. We propose the non-asymptotic technique for constructing a fixed-size confidence region for unknown parameters with any prescribed coverage probability. The construction makes use of some new properties of the sequential point estimates known in the literature. The results of Monte Carlo simulations for AR(1) and TAR(1) models are given. A new version of the sequential point estimate is proposed.