Skip to content

IBM: wall conduction through the unbounded mirror temperature for chemistry walls - #2248

Open
sbryngelson wants to merge 7 commits into
MFlowCode:masterfrom
sbryngelson:fix/isothermal-ghost-conduction-v2
Open

sbryngelson wants to merge 7 commits into
MFlowCode:masterfrom
sbryngelson:fix/isothermal-ghost-conduction-v2

Conversation

@sbryngelson

@sbryngelson sbryngelson commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Two commits on origin/master 478a270: the code change with the new restart test, then the regenerated goldens. Without chemistry nothing changes (every IBM-labelled test passes with its existing golden on both vendors); the new code is gated on ib, chemistry and chem_params%diffusion. No parameter is added.

Summary

For a chemistry immersed boundary the ghost temperature was one number, the linear mirror T_g = T_s + theta (T_s - T_IP), held inside the 200-5000 K window of the thermodynamic fits, because the chemistry diffusion flux also evaluated the ghost cell's conductivity, diffusivities, enthalpies and density at it. For a cold wall in hot gas the window cuts theta hard (T_s = 300 K, T_IP = 1500 K gives theta <= 0.9 (300 - 200)/(1500 - 300) = 0.075), so the wall heat flux is carried by a gradient about 13 times too small, and an under-resolved layer could still push the first gas cell past T_s.

  • Conduction gradient (m_ibm.fpp, m_rhs.fpp). The mirror is stored per ghost (ghost_point%T_cond, s_ibm_set_conduction_temperature) and put in q_T_sf at the ghost, with no window, only for the face fluxes. The ghosts' own temperatures are put back right after (s_ibm_restore_conduction_temperature), so the reaction source and the split chemistry substep see them unchanged. theta is cut by f_face_theta so that the two-point conduction across a face to a fluid neighbour cannot take that neighbour past T_s; a neighbour within a quarter cell of the wall plane is left out (its temperature is within noise of T_s).
  • Properties at the ghost's own state (m_chemistry.fpp). The diffusion flux evaluates the properties of an immersed-body cell at its own state temperature p W/(rho R) (ib_mode 1 and 2); only the face temperature difference comes from q_T_sf. use_wall becomes the integer ib_mode (0 no body, 1 body, 2 body with ib_flux_wall); the diffusion kernels keep one GPU_PARALLEL_LOOP per mode.
  • Advected ghost (m_ibm.fpp). It is wall gas at T_s for a fixed body. A moving body's lab-frame faces advect the ghost into the fluid, so it carries the windowed mirror.
  • Restart. The conduction temperature is not checkpointed: the first stage after a restart conducts through the ghost's own temperature. The lumped-wall restart round trip is therefore a new test with diffusion off.

GPU pattern: the new kernels store only scalar components (ghost_points(i)%T_cond, %T_save), never whole-array assignments; the new device routines are seq routines with scalar arguments, written like the existing s_blend_ghost_temperature.

Evidence

Machines. A100: NVIDIA A100, nvfortran 25.11, OpenACC, 1 rank. MI300A: AMD MI300A, amdflang (AFAR), OpenMP offload, 1 rank.

Measured demonstration (A100, master vs this branch). A fixed isothermal cylinder (thermal_bc = 1, Twall = 300 K, R = 1 mm) in quiescent 1500 K air (11-species mechanism of the reacting-surface example, diffusion on, reactions off, inviscid) in a closed box of half-width 3R. The table is the heat taken by the body per unit depth, the time integral of ib_heat_wrt column 5, up to t = 0.02 R^2/alpha = 59.2 us (alpha of the 1500 K gas; the conduction layer is about 0.14 R thick). The reference is this branch's 128 cells/R run, which is itself not fully converged (it is not an independent solution), so the percentages show the rate of convergence, not an error bar.

cells / R master [J/m] master vs ref branch [J/m] branch vs ref
6 0.1617 -55.6 % 0.2361 -35.2 %
8 0.1912 -47.5 % 0.2378 -34.7 %
16 0.2539 -30.3 % 0.3057 -16.0 %
32 0.3040 -16.5 % 0.3387 -7.0 %
64 0.3388 -6.9 % 0.3527 -3.1 %
128 0.3611 -0.8 % 0.3641 ref

At every resolution the branch takes more heat; its heat at N cells/R is about what master reaches at 2N. At t = 0.1 R^2/alpha (296 us) the same ordering holds (8, 16, 32 cells/R, master then branch, J/m): 0.6009 / 0.6848, 0.7046 / 0.7709, 0.7774 / 0.8029; master at 64 cells/R is 0.8168 (the branch run at 64 cells/R was not finished). The two schemes converge to the same heat; the change is in how fast. The coldest gas cell outside the body stayed above Twall in every run (lowest value 335 K), so this case does not show the first-gas-cell overshoot that the face bound f_face_theta guards against; I have no measured case for that guard beyond the unchanged-answer tests below.

Tests

./mfc.sh format and ./mfc.sh precheck pass. Labels run separately and together, --no-build after a dry-run build, with the master goldens first:

Label A100 MI300A
IBM 85 / 85 pass 85 / 85 pass
IBM and Chemistry (both labels) 1 / 1 pass 1 / 1 pass
Chemistry, master goldens 32 pass, 10 fail 32 pass, 10 fail

The same ten tests fail on both vendors. Nine are existing goldens that move (before the example 2D_ibm_lumped_wall was dropped from the suite in #2215; eight remain); the tenth, 2D -> Chemistry -> IBM Reacting Surface -> Lumped Wall -> Restart, is new and has no golden. The goldens were generated on the A100 node and checked on the MI300A: all ten pass there with the existing tolerances, so no override_tol is added. The new restart test also passes its restart check (straight run against restart from the first save, Twall included) on both. After merging master (#2200, #2213, and the thermostat golden fix #2251) five more goldens move, so thirteen move in all: 3F5FE078, 8B2A7F3E and FAF04D7A (the inert and trace-species flux walls of #2213) and F8D466BD and C02C399E (the thermostat tests of #2200/#2205). They were regenerated on the A100 node and pass on the MI300A at their existing tolerances. On the merged tree the full Chemistry label on the A100 fails exactly those five and the IBM label passes 102 of 102.

Which goldens move and by how much (largest change over all saved files of the test, divided by the largest value of that file; the golden files include the cells inside the body):

UUID Test density energy
F52F0D4C Example ibm_reacting_surface 1.7e-1 4.0e-1
40A4A5B8 Surface Output 1.1e-1 4.0e-1
A74B7381 Thermal Layer 9.1e-2 3.6e-1
C1F0BCB1 Flux Wall 1.1e-1 4.0e-1
F7A6649D Heat Output 1.1e-1 4.0e-1
641CC3F0 Lumped Wall 1.1e-1 4.0e-1
F389CE57 Lumped Wall, Thermal Layer 9.0e-2 3.6e-1
7E50FFE6 Lumped Wall, Radiation 1.1e-1 3.1e-1
8B2A7F3E Inert Surface, Flux Wall 1.1e-1 3.1e-1
FAF04D7A Inert Surface, Flux Wall, Closed Box 1.3e-1 3.1e-1
3F5FE078 Flux Wall, Trace Species 1.1e-1 4.1e-1
F8D466BD Thermostat 1.1e-1 4.0e-1
C02C399E Thermostat, Wall Temperature 1.1e-1 4.0e-1

Every one of these is an isothermal, reacting or lumped chemistry wall with diffusion on, which are the cases whose ghost temperature changed: the ghost cell now holds wall gas at T_s (density p W/(R T_s), internal energy at T_s) instead of the windowed mirror, so the ghost cells themselves change by tens of percent. In the 25 x 25 tests (all but the two examples) about 24 of 625 cells change by more than 1 % in density at the last save, which is the ring of ghost cells around the body; the rest of the field moves by less (more than 1e-4 relative in about 520 cells at the last save, presumably the response to the changed wall flux; I did not trace it). The heat history D/ib_heat.dat of the adaptive-step tests has a different row count (for example 120 to 114 rows) because the step count follows the changed wave speeds. I did not separate the change in the fluid cells alone from that in the ghost cells beyond this count. The five added goldens behave the same way: 24 ghost-ring cells (25 x 25 grids) change by more than 1e-2 of the field maximum; the ghost internal energy drops from up to 1.4e6 J/kg (the mirror overshoot) to 5.6e5 J/kg (the gas at T_s), the fluid cells next to the body stay between that and the freestream value, and no species density is negative. In the closed box the gas mass outside 7 cells from the axis is 0.12 % above its initial value before and 0.07 % after.

Limitations and not run

  • Not run: Cray CCE, GNU, Intel, CPU builds, more than one rank, 3D bodies, moving bodies (the windowed mirror for a moving wall's advected ghost is exercised by no test I added), thermal_bc = 2 outside the listed tests, AMR with bodies.
  • The conduction temperature is not checkpointed (above). A restart of a chemistry wall with diffusion on differs from the straight run for the first stage.
  • No timing was taken. f_face_theta adds a loop over the fluid neighbours of each ghost point in every stage.
  • The demonstration is a self-convergence study on one geometry and one temperature ratio; it is not a comparison with a body-fitted solution.

This PR was made with Claude Code (AI).

The ghost temperature of a chemistry immersed boundary was one number, the mirror T_g = T_s + theta (T_s - T_IP) held inside
the 200-5000 K window of the thermodynamic fits, because the diffusion flux also evaluated the ghost's conductivity,
diffusivities, enthalpies and density at it. For a cold body in hot gas the window cut theta to about 0.08 and with it the wall
heat flux, and an under-resolved layer could still drive the first gas cell past T_s.

- The conduction gradient reads the mirror in q_T_sf at the ghost (ghost_point%T_cond, s_ibm_set_conduction_temperature), with
  no window; the ghosts' own temperatures are put back once the face fluxes are formed. theta is cut by f_face_theta so the
  two-point conduction across a face to a fluid neighbour cannot take it past T_s; a neighbour within a quarter cell of the
  wall plane is left out, as its temperature is within noise of T_s.
- The chemistry diffusion evaluates the properties of an immersed-body cell at its own state temperature p W/(rho R) (ib_mode
  1 and 2); the face difference alone comes from q_T_sf.
- The advected ghost is wall gas at T_s for a fixed body; a moving body's lab-frame faces advect the ghost into the fluid, so
  it carries the windowed mirror.

The conduction temperature is not checkpointed: the first stage after a restart conducts through the ghost's own temperature.
The restart round trip of the lumped wall is therefore run with diffusion off.
… changes; add the lumped-wall restart golden

Nine chemistry immersed-boundary goldens move because the ghost state of an isothermal or reacting wall is now wall gas at T_s
and the conduction reads the unwindowed mirror. The lumped-wall restart round trip is a new test with diffusion off, since
the conduction temperature is not checkpointed. Generated on an A100 node (NVHPC, OpenACC) and checked on MI300A (amdflang,
OpenMP offload) with no tolerance change.
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Claude Code Review

Head SHA: b68928c

Files changed:

  • 7
  • docs/documentation/case.md
  • src/common/m_chemistry.fpp
  • src/common/m_constants.fpp
  • src/common/m_derived_types.fpp
  • src/simulation/m_ibm.fpp
  • src/simulation/m_rhs.fpp
  • toolchain/mfc/test/cases.py

Findings:

  • medium src/simulation/m_rhs.fpp: s_ibm_set_conduction_temperature is called under ib .and. chemistry .and. chem_params%diffusion, but the matching s_ibm_restore_conduction_temperature also requires .not. igr. If the set call sits after the conversion for both the IGR and non-IGR paths, an IGR run with IBM, chemistry and diffusion overwrites ghost cells in q_T_sf with the mirrored T_cond (which can be below 0 K) and never restores them. The reaction source and split chemistry would then see that temperature. I could not see the surrounding m_rhs code, so I am not certain. Use the same guard on both calls, or confirm that the set call is only reached on the non-IGR path.
  • medium src/simulation/m_ibm.fpp: T_cond is written in s_ibm_correct_state and stored on ghost_points, but it is not checkpointed. The new test comment confirms this: the first stage after a restart conducts through the ghost's own temperature, so a restarted run is not bit-consistent with a straight run whenever diffusion is on. The test only avoids this by turning diffusion off for the restart case. This is a known limitation, so confirm it is acceptable. At minimum, document it, since the restart_check coverage for the diffusion path is dropped.
  • medium src/simulation/m_ibm.fpp: f_face_theta computes T_n as p*MW*q_prim(adv%beg)/(q_prim(cont%beg)*R). The extra adv%beg factor is alpha. It is correct only for single-fluid chemistry where alpha = 1, whereas f_property_temperature in m_chemistry omits it. Consider dropping the factor so both temperature reconstructions are the same. I am fairly sure it is harmless today.

@sbryngelson
sbryngelson marked this pull request as ready for review November 14, 2026 03:20
@sbryngelson
sbryngelson requested a review from a team as a code owner November 14, 2026 03:20
@sbryngelson
sbryngelson requested review from danieljvickers and removed request for a team November 14, 2026 03:20
@codecov

codecov Bot commented Nov 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 60.00000% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.99%. Comparing base (2224b0c) to head (b68928c).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
src/common/m_chemistry.fpp 38.46% 14 Missing and 2 partials ⚠️
src/simulation/m_ibm.fpp 73.91% 3 Missing and 9 partials ⚠️
src/simulation/m_rhs.fpp 33.33% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2248      +/-   ##
==========================================
- Coverage   65.15%   64.99%   -0.17%     
==========================================
  Files         110      110              
  Lines       29555    29610      +55     
  Branches     4476     4488      +12     
==========================================
- Hits        19256    19244      -12     
- Misses       6743     6803      +60     
- Partials     3556     3563       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

Lines of Code

File Lines Diff
src/simulation/m_ibm.fpp 2957 +79
src/common/m_chemistry.fpp 524 +21
src/simulation/m_rhs.fpp 2089 +3
src/common/m_derived_types.fpp 543 +2
src/common/m_constants.fpp 105 +1
Directory Lines Diff
common 11071 +24
simulation 39560 +82
total 59550 +106

…FlowCode#2205

MFlowCode#2200 added the Thermostat and Thermostat -> Wall Temperature tests with goldens made before MFlowCode#2205
(second-order face composition in the diffusion correction flux) landed, and its CI ran on a base
without MFlowCode#2205. On master the two tests miss their goldens by up to 7.7e-6 relative (tolerance 1e-6).
…s ghost temperature also moves

8B2A7F3E, FAF04D7A and 3F5FE078 (MFlowCode#2213 inert and trace-species flux walls) and
F8D466BD and C02C399E (MFlowCode#2200 thermostat, goldens regenerated for MFlowCode#2205)
were generated on master without the isothermal ghost change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant