Repository navigation
Conversation
5446f0c to
a5690b9
Compare
Aman-Mittal
left a comment
There was a problem hiding this comment.
Reviewed the top commit. This is the step everything else was building towards and it holds together. Three inline questions: the skipHooks leak into the error phase, what the startup guard actually verifies, and malformed JSON on the capture path.
Things I traced and found right, recorded so they do not get re-litigated:
- Hook ordering is correct and load-bearing. 10/11/12 core, 20 key header, 21 claim, 30 audit. The claim therefore writes the row before
AuditCommandHookBeforewould, and becauseclaimsetscommand.setCommandId(...), the audit hook updates that row instead of inserting a second one. Reorder any of these and the unique constraint starts rejecting legitimate first requests. Worth saying so next to the constants, since the numbers live in three different modules. - Replaying through
ctx.setResponse(...)rather than through captured text is the right call, and the reason given (the same serializer renders both, so they match because they are the same object, not because two mappers were made to agree) is the only version of this that survives contact withBigDecimal. - Reading the key from the header only is correct. A key in the query string would end up in access logs and proxy logs.
425 Too Earlyfor a live claim,409for a key reused with a different request matches the legacy behaviour, andmatchesStoredRequestbeing checked before theERRORandPROCESSEDbranches is the right order: a mismatched request must not get the stored failure either.
One question not tied to a line: step 2 reinstates MaskedValueSerializer for the Jackson 3 mapper, so a command carrying credentials stores them masked. If such a command ever reaches this framework, matchesStoredRequest compares masked text against masked text, and two different secrets that mask identically would count as the same request and be replayed. No command in the stack has a masked field today, so this is forward-looking, but it is the one place where masking and request matching pull against each other. Is the intent that masked fields are never part of a command payload?
Command hooks now receive a CommandContext - the command, its response, its error and its state - instead of separate arguments, so a before hook can mark a command as handled (skipExecution) and have the dispatcher return a response it supplied, and can stop the remaining hooks (skipHooks). The synchronous, async and disruptor dispatchers all honour it; the disruptor dispatcher hands each command a fresh future before publishing its slot. Each command module gets constants for its properties, and the hook switches move under the module they belong to (fineract.command.audit.hook-*-enabled, fineract.command.hook-*-enabled). The audit module reports at startup which of its hooks are on. The JDBC store is renamed to its Jdbc*/Json* types and stores a CommandContext. The webhook command hooks are adapted to the new hook interface. Step 1 of 9 of FINERACT-2485. Co-authored-by: Aleksandar Vidakovic <aleks@apache.org>
…ormat The command store runs on Jackson 3 while Jersey can only run on Jackson 2 (jersey-media-json-jackson 4.0.2 has no Jackson 3 support). Jackson3ObjectMapperConfig defines the application's Jackson 3 mapper to produce the same JSON as the web mapper: the same date, time, ExternalId and Month formats, NON_NULL inclusion, and the two date-time defaults Jackson 3 turned off (WRITE_DATES_AS_TIMESTAMPS, WRITE_DURATIONS_AS_TIMESTAMPS). java.time.Month is written by name, because Jackson 3 writes it as a number that a Jackson 2 client reads back as the wrong month. Credential masking is reapplied: the Jackson 2 MaskedValueSerializer annotation is invisible to Jackson 3. JacksonWireFormatEquivalenceTest pins the agreement, and Jackson3ObjectMapperWiringTest checks that this mapper, not Spring Boot's default, is the one injected. Step 2 of 9 of FINERACT-2485.
UpdateWorkingDaysCommandHandler was the only command handler executing without a retry, so it gave up on the first contention where every other command tries again. It gets its own commandWorkingDaysUpdate instance, configured like the others. The product mix and floating rate handlers named retry instances that were never configured, so they fell back to resilience4j's defaults and retried on every exception. Their instances are configured like the others, and the unused commandMixTaxonomyMappingUpdate instance, left behind when the mix module was removed, is dropped. CommandHandlerRetryCoverageTest walks every CommandHandler on the classpath and fails if one is added without a retry, or names a retry instance that is not configured. Step 3 of 9 of FINERACT-2485.
Every command declares its entity and action names, in the vocabulary of m_permission, and m_command stores them in two new NOT NULL columns (rows written before are backfilled as UNKNOWN). The audit search and the checker inbox are defined in these terms, and maker-checker derives the approval permission from them, so a row without them is invisible to both. Declared as abstract methods on Command rather than an optional interface: a command that forgot them would be stored with none. The names follow m_permission, which is not always what the class name suggests (CACHE/UPDATE, BUSINESS_DATE, COLLATERAL_PRODUCT, CHARTSLAB, MEETING for attendance); notes and images compute the entity from their payload. HookableCommand becomes a marker, and a command read back from the store is a StoredCommand carrying the recorded names. Step 4 of 9 of FINERACT-2485.
The command store can now claim an idempotency key atomically and report what was stored for it, which the idempotency module in the next step builds on. - m_command.custom_idempotency_distinctor (NOT NULL, default '') holds what a key is unique together with, and a unique constraint on (idempotency_key, custom_idempotency_distinctor) makes a claim atomic. Rows from earlier builds that share a key keep the earliest one; the others get their id appended, so no record is deleted. - m_command.response_status_code records the status of an answer that is not the command's own result (a failure, or an answer that is not JSON). - CommandStore gains claim(), getReplayByKey(), matchesStoredRequest() and recordResponse(); every lookup by key also takes the distinctor, and IdempotencyKeyLookupCoverageTest fails the build on one that does not. A stored result that cannot be read back fails loudly instead of answering null. - Writes of the command row are retried on transient and recoverable data-access failures; a constraint violation is not retried, because the row is already there. Until the idempotency module is added, a second request with a key that is already stored is rejected by the constraint instead of being executed again. Step 5 of 9 of FINERACT-2485. Co-authored-by: Aleksandar Vidakovic <aleks@apache.org>
A repeat of an idempotency key is answered by reading the command's stored result back into the object it was. Four collateral command results (client collateral create and update, collateral product update, loan collateral delete) had a builder but no constructor Jackson could use, so they could be stored but not read back. The collateral responses get no-args and all-args constructors. CommandHandlerResultRehydrationTest walks every CommandHandler on the classpath and reads its result type back the way a replay does, with every nested type present, so a new result type cannot regress this silently. CommandReplayRehydrationTest checks that a result read back renders byte for byte as the original. Step 6 of 9 of FINERACT-2485.
The new fineract-command-idempotency module answers a repeated request the way the legacy processing does: - The key is read from the Idempotency-Key header (header only, never a query parameter) and claimed before the command runs, so only one of several concurrent requests executes. - A repeat of a completed command is answered with the command's own result, read back from the store and rendered by the same serializer; a repeat of a failed one with the original failure; a repeat of one still running with 425 Too Early. Every replay carries x-served-from-cache: true. A key reused with a different request is refused with 409. - An answer that is not the command's own result is captured by a response filter, which logs the key, status, content type and size of an answer it cannot store, never its body. - A key is unique together with a distinctor, chosen by fineract.command.idempotency.custom-distinctor: ENTITY_NAME_AND_ACTION_NAME (default, e.g. LOAN|CREATE) scopes it as the legacy processing does, NONE makes it unique per tenant. - The audit hooks move a claimed command out of UNDER_PROCESSING, so an application with idempotency on refuses to start with auditing, or its after or error hook, switched off. The key header moves from fineract.command.idempotency-key-header-name to fineract.command.idempotency.key-header-name, keeping FINERACT_IDEMPOTENCY_KEY_HEADER_NAME. New command-auditing chapter; the idempotency chapter describes the new processing. Step 7 of 9 of FINERACT-2485. Co-authored-by: Aleksandar Vidakovic <aleks@apache.org>
a5690b9 to
7730742
Compare
|
Two points from the review summary:
|
Step 7 of 9 of FINERACT-2485 — splitting #6871 into reviewable pieces. Merge in step order.
This PR targets
develop, so until the earlier steps merge it also shows their commits. Review only the top commit — the changes of this step: compare with step 6. Depends on #6912.What this step does
The new
fineract-command-idempotencymodule answers a repeated request the way the legacy processing does.Idempotency-Keyheader (header only) and claimed before the command runs: only one of several concurrent requests executes.425 Too Early. Every replay carriesx-served-from-cache: true. A key reused with a different request →409.fineract.command.idempotency.custom-distinctor:ENTITY_NAME_AND_ACTION_NAME(default, e.g.LOAN|CREATE, as legacy) orNONE(unique per tenant). An application can supply its ownCustomIdempotencyDistinctor.UNDER_PROCESSING, so with idempotency on the application refuses to start if auditing, or its after or error hook, is off.Configuration change:
fineract.command.idempotency-key-header-namebecomesfineract.command.idempotency.key-header-name(sameFINERACT_IDEMPOTENCY_KEY_HEADER_NAMEvariable). Adds the command-auditing doc chapter and the new-processing section of the idempotency chapter.The module structure, hooks and exceptions come from #6231 by Aleksandar Vidakovic (credited as co-author); this step adds replay from the result object, the response capture, the distinctor and the startup guard.
Verified
fineract-command-idempotencybuild with unit and end-to-end tests; provider idempotency tests; command module builds. Not verified on a running Fineract instance.Stack
Umbrella and history: #6871.