Describe the bug

When using MapieRegressor to generate prediction intervals, an intermittent error occurs with the following message:

ValueError: The two functions get_conformity_scores and get_estimation_distribution of the ConformityScore class are not consistent. The following equation must be verified: self.get_estimation_distribution(y_pred, self.get_conformity_scores(y, y_pred)) == y. The maximum conformity score is 3.0517578125e-05. The eps attribute may need to be increased if you are sure that the two methods are consistent.

The cause of this error and its inconsistency is currently unknown.

To reproduce

The error occurs approximately 15% of the software runs. Despite multiple attempts to reproduce the error using the provided code, identical data, Python version, and installed packages, the issue could not be replicated. In other words, the error appears to be specific to my software and is not observed in other environments.

import pandas as pd
from mapie.regression import MapieRegressor
from sklearn import linear_model
from sklearn.exceptions import ConvergenceWarning
from sklearn.model_selection import train_test_split
from tqdm import tqdm

data = pd.read_csv("data/autompg.csv")
n = 1000
target = "mpg"

for _ in tqdm(range(n)):
    train, calib = train_test_split(data, test_size=0.25, shuffle=True)

    X_train, y_train = train.drop(columns=[target]), train[target]
    X_calib, y_calib = calib.drop(columns=[target]), calib[target]

    model = linear_model.LinearRegression()
    model.fit(X_train, y_train)

    mapie = MapieRegressor(model, cv="prefit")
    mapie.fit(X_calib, y_calib)

Expected behavior

I would like to understand the circumstances under which this exception is raised. Additionally, I seek guidance on how to resolve the issue.

Desktop

• OS: Ubuntu 20.04.6 LTS
• MAPIE version: 0.6.4
• Python 3.7 (although the issue also occurred with 3.10)