We combine a consistent (base) estimator of a population parameter with one or several other possibly inconsistent estimators. Some or all assumptions used for calculating the latter estimators may be incorrect. The suggested in the manuscriptapproach is not restricted to parametric families and can be easily used for improvingefficiency of estimators built under nonparametric or semiparametricmodels. The combined estimator minimizes the mean squared error (MSE) in a family of linear combinations of considered estimators when all variances and covariancesused in its structure are known. In real life problems these variances and covariances are estimated generating an empirical version of the combined estimator.The combined estimator as well as its empirical version are consistent. The asymptotic properties of these estimators are presented. The combined estimator is applicable when analysts can use several different procedures for estimating the same population parameter. Different assumptions are associated with the use of each of non-base estimators. Our estimator is consistent in the presence of wrongassumptions for non-base estimating procedures. In addition to theoretical resultsof this manuscript, simulation studies describe properties of the estimator combiningthe Kaplan-Meier estimator with the censored data exponential estimator of a survival curve. Another set of simulation examples combine semi-parametricCox regression with exponential regression on right censored data.