Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suppressing @info statement during construction of AtmosSimulation #3658

Open
glwagner opened this issue Feb 26, 2025 · 6 comments
Open

Suppressing @info statement during construction of AtmosSimulation #3658

glwagner opened this issue Feb 26, 2025 · 6 comments

Comments

@glwagner
Copy link
Member

glwagner commented Feb 26, 2025

I'd be nice to design a way to suppress @info statements during the construction of AtmosSimulation. In a scripting context it is nice for users to be able to pick the output they desire; ie @show simulation, or properties thereof.

In a development (or coupling) context the extensive output becomes a hindrance rather than merely a distraction: the long output drowns out important warnings and errors that occur on package import. As a result, developing packages that depend on ClimaAtmos is difficult.

This is the output I get with a fairly simple configuration (I think):

┌ Warning: Failed to convert `config_dict["vert_diff"] = DecayWithHeightDiffusion` to default type Bool, keeping original value
└ @ ClimaAtmos ~/Projects/ClimaAtmos.jl/src/solver/yaml_helper.jl:80
┌ Info: Making AtmosConfig with config files:/Users/gregorywagner/Projects/ClimaAtmos.jl/src/solver/../../config/default_configs/default_config.yml
└    simple_atmos_simulation.yml
[ Info: Running on CPUMultiThreaded
[ Info: Setting up single-process ClimaAtmos run
┌ Info: Time info:
│   dt = "400secs"
│   t_end = "1days"
└   floor_n_steps = 216
┌ Warning: nothing does nothing if RRTMGP is not used
└ @ ClimaAtmos ~/Projects/ClimaAtmos.jl/src/solver/type_getters.jl:45
[ Info: Using CAM_SE hyperdiffusion. vorticity_hyperdiffusion_coefficient, scalar_hyperdiffusion_coefficient and divergence_damping_factor in the config will be ignored.
┌ Info: numerics
│            `energy_upwinding`::`Val{:vanleer_limiter}``tracer_upwinding`::`Val{:vanleer_limiter}``edmfx_upwinding`::`Val{:none}``edmfx_sgsflux_upwinding`::`Val{:none}``test_dycore_consistency`::`Nothing``limiter`::`Nothing`
┌ Info: AtmosModel:
│
│                       `moisture_model`::`ClimaAtmos.EquilMoistModel``precip_model`::`ClimaAtmos.Microphysics0Moment``cloud_model`::`ClimaAtmos.GridScaleCloud``call_cloud_diagnostics_per_stage`::`Nothing``forcing_type`::`Nothing``subsidence`::`Nothing``ozone`::`ClimaAtmos.IdealizedOzone``co2`::`Nothing``radiation_mode`::`Nothing``ls_adv`::`Nothing``external_forcing`::`Nothing``edmf_coriolis`::`Nothing``tendency_model`::`ClimaAtmos.UseAllTendency``edmfx_model`::`ClimaAtmos.EDMFXModel{ClimaAtmos.NoEntrainment, ClimaAtmos.NoDetrainment, Val{false}, Val{false}, Val{false}, Val{false}}``turbconv_model`::`Nothing``non_orographic_gravity_wave`::`Nothing``orographic_gravity_wave`::`Nothing``hyperdiff`::`ClimaAtmos.ClimaHyperdiffusion{Float32}``vert_diff`::`ClimaAtmos.DecayWithHeightDiffusion{true, Float32}``diff_mode`::`ClimaAtmos.Implicit``sgs_adv_mode`::`ClimaAtmos.Explicit``viscous_sponge`::`Nothing``smagorinsky_lilly`::`Nothing``rayleigh_sponge`::`ClimaAtmos.RayleighSponge{Float32}``sfc_temperature`::`ClimaAtmos.ZonallySymmetricSST``insolation`::`ClimaAtmos.IdealizedInsolation``surface_model`::`ClimaAtmos.PrescribedSurfaceTemperature``surface_albedo`::`ClimaAtmos.ConstantAlbedo{Float32}``numerics`::`ClimaAtmos.AtmosNumerics{Val{:vanleer_limiter}, Val{:vanleer_limiter}, Val{:none}, Val{:none}, Nothing, Nothing}`
┌ Info: Simulation info
│   job_id = "simple_atmos_simulation"
└   output_dir = "output/simple_atmos_simulation/output_0023"
┌ Info: z heights
│   z_mesh.faces =32-element Vector{ClimaCore.Geometry.ZPoint{Float32}}:ZPoint(0.0)
│     ZPoint(50.0)
│     ZPoint(111.19333)
│     ZPoint(186.07025)
│     ZPoint(277.6673)
│     ZPoint(389.68335)
│     ZPoint(526.6181)
│     ZPoint(693.93726)
│     ZPoint(898.2667)
│     ZPoint(1147.6201)
│     ZPoint(1451.6614)
│     ZPoint(1822.0024)
│     ZPoint(2272.5344)
│     ZPoint(2819.7852)
│     ZPoint(3483.2854)
│     ZPoint(4285.923)
│     ZPoint(5254.238)
│     ZPoint(6418.6025)
│     ZPoint(7813.2017)
│     ZPoint(9475.693)
│     ZPoint(11446.42)
│     ZPoint(13767.015)
│     ZPoint(16478.242)
│     ZPoint(19616.984)
│     ZPoint(23212.363)
│     ZPoint(27281.178)
│     ZPoint(31823.062)
│     ZPoint(36816.08)
│     ZPoint(42213.59)
│     ZPoint(47943.434)
│     ZPoint(53910.074)
└     ZPoint(60000.0)
[ Info: Remapping Earth orography from ETOPO2022 data onto horizontal space
[ Info: Linear mesh warp
┌ Info: Resolution stats:
│   Nq = 4
│   h_elem = 6
│   z_elem = 31
│   ncols = 3456
└   ndofs_total = 107136
[ Info: Allocating Y: 761.887 ms (3522994 allocations: 275.19 MiB)
[ Info: using deep atmosphere
[ Info: Allocating cache (p): 6.826 s (63114931 allocations: 3.66 GiB)
[ Info: Using ODE config: `ClimaTimeSteppers.ARS343`
[ Info: ode_configuration: 322.954 ms (1247294 allocations: 76.36 MiB)
[ Info: Progress logging enabled
[ Info: Checking NaNs in the state every 1024 steps
[ Info: get_callbacks: 97.493 ms (263770 allocations: 18.26 MiB)
┌ Warning: Disabling vertical interpolation, the provided number of points is ignored (using 31)
└ @ ClimaDiagnostics.Writers ~/.julia/packages/ClimaDiagnostics/K1oj5/src/netcdf_writer.jl:144
[ Info: Saving accumulated diagnostics to disk with frequency: 1 day
[ Info: DictWriter: String[]
[ Info: HDF5Writer: String[]
[ Info: NetCDFWriter: ["orog_inst", "ts_1d_average", "ta_1d_average", "tas_1d_average", "uas_1d_average", "vas_1d_average", "thetaa_1d_average", "ha_1d_average", "pfull_1d_average", "zg_1d_average", "rhoa_1d_average", "ua_1d_average", "va_1d_average", "wa_1d_average", "hfes_1d_average", "ts_1d_min", "ts_1d_max", "hur_1d_average", "hus_1d_average", "cl_1d_average", "clw_1d_average", "cli_1d_average", "hussfc_1d_average", "evspsbl_1d_average", "pr_1d_average", "prra_1d_average", "prsn_1d_average", "prw_1d_average", "lwp_1d_average", "clwvi_1d_average", "clivi_1d_average"]
[ Info: initializing diagnostics: 2.909 s (14307966 allocations: 989.62 MiB)
[ Info: Prepared SciMLBase.CallbackSet callbacks: 3.972 ms (408 allocations: 29.27 KiB)
[ Info: n_steps_per_cycle_per_cb (non diagnostics): [1024, 1, 27]
[ Info: n_steps_per_cycle (non diagnostics): 27648
[ Info: Define ode function: 2.326 s (16006610 allocations: 1000.7 MiB)
[ Info: dt_save_to_sol: 86400.0, length(saveat): 2
[ Info: init integrator: 4.936 s (30324338 allocations: 1.86 GiB)
[ Info: Added diagnostics: 5.378 s (33169192 allocations: 2.11 GiB)
@glwagner
Copy link
Member Author

yml file:

z_max: 60000.0
z_elem: 31
dz_bottom: 50.0
dt: "400secs"
t_end: "1days"
topography: "Earth"
rayleigh_sponge: true
implicit_diffusion: true
approximate_linear_solve_iters: 2
moist: "equil"
surface_setup: "PrescribedSurface" #DefaultMoninObukhov"
vert_diff: "DecayWithHeightDiffusion"
precip_model: "0M"
cloud_model: "grid_scale"
toml: [sphere_aquaplanet.toml]

@Sbozzolo
Copy link
Member

We already largely have this, in some way.

We use the Julia Logging system, so you can filter logs as you want.

Here's some docs https://clima.github.io/ClimaComms.jl/dev/logging/#How-do-I-filter-out-messages-from-certain-modules?

What we don't have is a simple way to pass the logger to the simulation

@glwagner
Copy link
Member Author

Ok I will see if I can experiment with this. We still want logging with @info, we merely want to suppress the ones from ClimaAtmos during construction. (Note, Oceananigans Simulation also will print things by default, but users can suppress them with verbose=false in the construction of Simulation).

I am not quite sure if what I would like to achieve can merely be achieved by filtering @info --- would this suppress all @info statements?

@Sbozzolo
Copy link
Member

Ok I will see if I can experiment with this. We still want logging with @info, we merely want to suppress the ones from ClimaAtmos during construction. (Note, Oceananigans Simulation also will print things by default, but users can suppress them with verbose=false in the construction of Simulation).

I am not quite sure if what I would like to achieve can merely be achieved by filtering @info --- would this suppress all @info statements?

We can suppress @info only from atmos (as shown at the bottom of the documentation page) (and atmos pretty much only uses @info in the init)

@nefrathenrici
Copy link
Member

nefrathenrici commented Feb 28, 2025

I agree, it would be nice to have a simple verbose option. For now, you can use this snippet to filter out logs from individual modules

using Logging, LoggingExtras, ClimaAtmos

filter_climaatmos = log_args -> log_args._module != ClimaAtmos

logger = EarlyFilteredLogger(filter_climaatmos, Logging.current_logger())
global_logger(logger)

config = ClimaAtmos.AtmosConfig()
ClimaAtmos.get_simulation(config)

@glwagner
Copy link
Member Author

glwagner commented Mar 6, 2025

Getting an error, with

import ClimaAtmos as Atmos
using Printf

# Making ClimaAtmos a little less chatty
using Logging
using LoggingExtras

filter_climaatmos = log_args -> log_args._module != ClimaAtmos
logger = EarlyFilteredLogger(filter_climaatmos, Logging.current_logger())
global_logger(logger)

output_prefix = "aquaplanet"

config_dict = Dict(
    "device" => "CUDADevice",
    "z_max" => 60000.0,
    "z_elem" => 31,
    "h_elem" => 12, # h_elem = 30 => ~ 1 degree
    "dz_bottom" => 50.0,
    "dt" => "200secs",
    "topography" => "NoWarp", # "Earth"
    "rayleigh_sponge" => true,
    "implicit_diffusion" => true,
    "approximate_linear_solve_iters" => 2,
    "moist" =>  "equil",
    "surface_setup" => "PrescribedSurface", #DefaultMoninObukhov"
    "vert_diff" => "DecayWithHeightDiffusion",
    "precip_model" =>  "0M",
    "cloud_model" =>  "grid_scale",
    "log_progress" =>  false,
    "output_dir" => output_prefix,
    "toml" => ["sphere_aquaplanet.toml"]
)

atmosphere = Atmos.AtmosSimulation(config_dict)

I get an error

Stacktrace:
  [1] (::var"#1#2")(log_args::@NamedTuple{level::Base.CoreLogging.LogLevel, _module::Module, group::Symbol, id::Symbol})
    @ Main /orcd/data/raffaele/001/glwagner/ClimaEarth.jl/examples/coupled_global_model/aquaplanet.jl:13
  [2] shouldlog(::EarlyFilteredLogger{ConsoleLogger, var"#1#2"}, ::Base.CoreLogging.LogLevel, ::Module, ::Symbol, ::Symbol)
    @ LoggingExtras /orcd/data/raffaele/001/glwagner/.julia/packages/LoggingExtras/cFgEq/src/CompositionalLoggers/earlyfiltered.jl:32
  [3] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
  [4] invokelatest
    @ ./essentials.jl:889 [inlined]
  [5] macro expansion
    @ ./logging.jl:394 [inlined]
  [6] override_default_config(config_dict::Dict{String, Any})
    @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/yaml_helper.jl:80
  [7] ClimaAtmos.AtmosConfig(configs::Tuple{Dict{String, Any}}; comms_ctx::Nothing, config_files::Vector{String}, job_id::String)
    @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/types.jl:736
  [8] AtmosConfig
    @ /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/types.jl:723 [inlined]
  [9] AtmosConfig
    @ /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/types.jl:721 [inlined]
 [10] ClimaAtmos.AtmosSimulation(args::Dict{String, Any}; kwargs::@Kwargs{})
    @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/simulation/AtmosSimulations.jl:40
 [11] ClimaAtmos.AtmosSimulation(args::Dict{String, Any})
    @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/simulation/AtmosSimulations.jl:39
 [12] top-level scope
    @ /orcd/data/raffaele/001/glwagner/ClimaEarth.jl/examples/coupled_global_model/aquaplanet.jl:51
 [13] include(fname::String)
    @ Base.MainInclude ./client.jl:494
 [14] top-level scope
    @ REPL[1]:1
in expression starting at /orcd/data/raffaele/001/glwagner/ClimaEarth.jl/examples/coupled_global_model/aquaplanet.jl:51

caused by: MethodError: Cannot `convert` an object of type String to an object of type Bool

Closest candidates are:
  convert(::Type{Bool}, ::LLVM.ConstantInt)
   @ LLVM /orcd/data/raffaele/001/glwagner/.julia/packages/LLVM/b3kFs/src/core/value/constant.jl:171
  convert(::Type{T}, ::VectorizationBase.AbstractSIMD) where T<:Union{Bool, Float16, Float32, Float64, Int16, Int32, Int64, Int8, UInt16, UInt32, UInt64, UInt8, SIMDTypes.Bit}
   @ VectorizationBase /orcd/data/raffaele/001/glwagner/.julia/packages/VectorizationBase/wHnQd/src/base_defs.jl:201
  convert(::Type{T}, ::VectorInterface.Zero) where T<:Number
   @ VectorInterface /orcd/data/raffaele/001/glwagner/.julia/packages/VectorInterface/Vk8YT/src/onezero.jl:83
  ...

Stacktrace:
 [1] override_default_config(config_dict::Dict{String, Any})
   @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/yaml_helper.jl:76
 [2] ClimaAtmos.AtmosConfig(configs::Tuple{Dict{String, Any}}; comms_ctx::Nothing, config_files::Vector{String}, job_id::String)
   @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/types.jl:736
 [3] AtmosConfig
   @ /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/types.jl:723 [inlined]
 [4] AtmosConfig
   @ /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/solver/types.jl:721 [inlined]
 [5] ClimaAtmos.AtmosSimulation(args::Dict{String, Any}; kwargs::@Kwargs{})
   @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/simulation/AtmosSimulations.jl:40
 [6] ClimaAtmos.AtmosSimulation(args::Dict{String, Any})
   @ ClimaAtmos /orcd/data/raffaele/001/glwagner/ClimaAtmos.jl/src/simulation/AtmosSimulations.jl:39
 [7] top-level scope
   @ /orcd/data/raffaele/001/glwagner/ClimaEarth.jl/examples/coupled_global_model/aquaplanet.jl:51
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:494
 [9] top-level scope
   @ REPL[1]:1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants