Extra.Forbid is not applied when using BaseSettings with Config.env_file defined

[jberthier-ma]

Initial Checks

• I have searched GitHub for a duplicate issue and I'm sure this is something new
• I have searched Google & StackOverflow for a solution and couldn't find anything
• I have read and followed the docs and still think this is a bug
• I am confident that the issue is with pydantic (not my code, or another library in the ecosystem like FastAPI or mypy)

Description

Reading the documentation and as BaseSettings inherits from BaseModel it'd be expected to have Config.extra parameters working when Config.env_file is defined.

It appears that without Config.env files it works as expected, but when it comes from env_files the extra config is ignored.

If this is expected I can make a PR to make the documentation more explicit on this specific point.

Example Code

from pydantic import BaseSettings, Extra

""""
# .env.test
title="cool stuff"
other=42
"""

class Plop(BaseSettings):
    title: str

    class Config:
        extra: Extra = Extra.forbid
        env_file = ".env.test"

Plop()

Python, Pydantic & OS Version

pydantic version: 1.10.2
            pydantic compiled: True
                 install path: .venv/lib/python3.11/site-packages/pydantic
               python version: 3.11.0 (main, Oct 24 2022, 19:55:51) [GCC 9.4.0]
                     platform: Linux-5.17.0-051700-generic-x86_64-with-glibc2.31
     optional deps. installed: ['dotenv', 'typing-extensions']

Affected Components

• Compatibility between releases
• Data validation/parsing
• Data serialization - .dict() and .json()
• JSON Schema
• Dataclasses
• Model Config
• Field Types - adding or changing a particular data type
• Function validation decorator
• Generic Models
• Other Model behaviour - construct(), pickling, private attributes, ORM mode
• Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

[samuelcolvin]

Good catch, thanks for reporting. Realistically this won't be changed in pydantic v1.10, but we should make sure it's fixed in pydantic V2 where settings will become a separate package which uses pydantic.

[hramezani]

@samuelcolvin Is it really a bug? It seems we loop over settings fields and collect the value for the fields from file or environment variables. So, we won't get the extra value provided in the file. Am I wrong?

[samuelcolvin]

I haven't actually tried it.

We need to check the current behaviour then decide if it's correct or needs changing.

[hramezani]

Currently we loop over fields and collect the values. If we are going to change the behavior, we need to collect the envs and then initialize the model. Then the default behavior(collect envs from environment without prefix and dotenv file) will be tricky because if we define a model with a field named test_field, then the other environment variables are extra. We may need to change it just for dotenv file or prefixed envs

[samuelcolvin]

Ye, extra behaviour should definitely always be "ignore" for env vars, but for other sources like dotenv, I can see the value in allowing "forbid" or "include".

[hramezani]

OK, that needs to be changed and documented

[jberthier-ma]

It's me who thanks you for all your amazing contributions to the python ecosystem !
Thanks for taking that behavior into account.