IDoc Tools, IDoc serialization

IDoc Tools

Inbound EDI Monitors in Business Function Set LOG_SD_EDIMON
WL_IDOC New IDoc Monitor

Serialization

IDoc serialization
Serialization using qRFC (as of 6.40)
Serialization using a time stamp
How does time stamp serialization work?
Checking time stamp serialization

Serialization of Object Types (serialization groups)

Serialization at IDoc Level
Serialization Using Message Types
Serialization in the sending system

Relationships
References

752194 - Serialization of IDoc processing
817939 - IDoc: IDocs remain in status 75
1862256 - Inbound queues (SMQ2) stay in status RETRY
1824405 - Inbound queue scheduler : customize Scheduler Monitoring
2556837 - How to manually restart Q/T RFC scheduler SMQS/SMQR
Analysis of the FM involved
1804549 - Restarting Inbound SYSFAIL, RETRY, CPIERR and RUNNING Queues

IDoc Tools

Inbound EDI Monitors in Business Function Set LOG_SD_EDIMON

EMORD is for inbound SD Orders (Basic Type = ORDER01 to ORDERS05, Logical type - ORDERS, ORDCHG, DELNOT).
EMFOR is for inbound SD Forecast and JIT Delivery Schedules (Basic type = DELFOR01 to DELFOR02, Logical type - DELFOR, DELJIT, DELINS)
EMJIT is for inbound SD Summarized and Sequenced JIT Calls (Basic type = SEQJIT01 to SEQJIT03, Logical Type - SEQJIT)
EMASN is for inbound MM Shipping Notifications (Basic Type = DELVRY01 to DELVRY07, Logical Type - DESADV)

There is a program called SDJEDI that can be run from transaction SE38. This program can be used to run the IDOC in foreground mode in simulation mode.

WL_IDOC New IDoc Monitor

Selection

Optimized and additional selection options

Selection options using all fields of the IDoc control record (replace WE02)
Criteria for data record (replace WE09)

Process-based selections - a predefined action is executed for the hits of a selection.

Monitoring: We have the option to trigger an error if the selection shows a specific result.
Background processing

Output: A large number of new functions are available on the output screen.

Direct display of linked objects
Display of current status record
Navigation to the message type, partner profiles, IDoc type, and so on
Send IDocs to another logical system.
Compare IDocs
Call up log
Change status
Process selected IDocs
Change control record
Copy IDoc and delete a segment

New IDoc processing functions

The new IDoc editor is a configurable tool

Customizing: SPRO->Cross-Application Components->IDOC Monitor for Agency Business and Retail->IDOC Maintenance Settings.

BAdIs WLF_CHANGE_BEFORE_IDOC_OUT_BD and WLF_REPORT_PROCESS_BD. We can use these BAdIs to fill added fields prior to the list output and add new actions.

Serialization

Serialization plays an important role in distributing interdependent objects, especially when master data is being distributed. IDocs can be created, sent, and posted in a specified order by distributing message types serially.

IDoc serialization

Use the Integration Server to process the corresponding IDoc XML messages in the same sequence that it receives them from the IDoc adapter at the inbound channel.
Use the receiver to receive the IDocs in the same sequence that the IDoc adapter sends them at the Integration Server outbound channel.

The quality of service EOIO (Exactly Once In Order) must be specified in the message header.

Enter a queue name in the application
The IDoc adapter checks the prefix and replaces it with the prefix of the corresponding Integration Server inbound queue (for example, XBQI0000).

Serialization using qRFC (as of 6.40)

As of Release 6.40, ALE can use qRFC. This method makes it possible to process all LUWs that were sent to a destination in the sequence in which they were created.

In the ALE environment, we can only use qRFC if the sending and receiving systems support this method.

To activate qRFC, we simply have to activate "queue processing" for the message types in question in the partner profiles (transaction: WE20).
In addition to this, we must define the rule that determines the queue name. The rules are defined in transaction WE85 and refer to a function module that returns the name of the queue when it is called.
We can use transactions WEINBQUEUE and WEOUTQUEUE to control the individual queues.

Serialization using a time stamp

We use this type of serialization only if a number of changes occur for an application document and if maintenance may lead to business inconsistencies. The system rejects changes that have an earlier time stamp, and this means that the system only processes the most current change.

Use the RBDSRCLR report to clean up this table regularly.

How does time stamp serialization work?

In the sending system, the system generally fills the SERIAL field for each IDoc in the control record of an IDoc with the current time stamp.
In the inbound function module, we must call the IDOC_SERIALIZATION_CHECK function module to check whether there is already an entry in the BDSER table.
If this is the case, the system has to check whether the time stamp of the IDoc is older than the entry from the BDSER table. If it is, the processing of the IDoc terminates with the error status '51'.

Checking time stamp serialization

In transaction BD57, we can check whether an ALE object type is assigned to the message type as a serialization object.
In BD59,we can check whether the serialization object is assigned to a segment field.

We can use the where-used list of the IDOC_SERIALIZATION_CHECK function module to determine the modules that allow time stamp serialization. If we do not call this function module, then there is no time stamp serialization for the object or scenario that is being checked.

Serialization of Object Types (serialization groups)

Use

Serialized messages can be of different types (for example, create, change or cancel messages). All messages here relate to one special application object. Messages are posted in the receiver system in the same order they were created in the sender system.

Example

When master data is distributed, interdependent objects are often distributed together (for example, purchasing info record with vendor and material).

Serialization groups in which the messages to be used and the posting order is specified, are used to distribute interdependent messages serially.

Tracking by IDoc status

An IDoc is waiting in inbound processing for its predecessor IDoc and is accordingly assigned status code 66. Once the predecessor IDoc has been posted, we have to then post the IDoc with status 66 using program RBDAPP01.

All messages with the same channel number and object type are serialized when the messages are processed.

Outbound Processing (Source System)

When outbound IDocs are processed, for each object channel (field CHNUM) a unique serial number is assigned to each IDoc created (field CHCOU). This number and the object channel are transferred with the IDoc in the SERIAL field.

Inbound Processing (Target System)

When inbound IDocs are processed, a unique serial number is generated for each object channel (field CHNUM) and communication link. The ALE layer determines whether a given IDoc can now be posted or whether other IDocs have to be posted first. The serial number for each received IDoc is exactly one whole number lower. Any gaps in the sequence will mean that IDocs are missing, either because the transfer did not work, or because earlier IDocs were not posted successfully.

Procedure:

IDoc Interface/ALE → Development → BAPI → Serialization → Serialization Using Business Objects → Determine Supported Business Objects (transaction BD105)
IDoc Interface/ALE → Development → BAPI → Serialization → Serialization Using Business Objects → Assign Message Type to a Business Object (transaction BD104)
In Customizing (IMG) activate the serialized distribution in both the sending and receiving systems: ALE Implementation Guide (transaction SALE) /Modeling and Implementing Business Processes/Master Data Distribution/Serialization for Sending and Receiving Data /Serialization Using Business Objects/ Execute activities Activate Outbound Business Objects and Activate Inbound Business Objects. Set the Serialization flag for the required business object types.

Define Serialization Groups

Assign Message Types to the Serialization Group

In addition to the message types used, the dispatch of the message type SERDAT must be modeled in the distribution model.
Define Inbound Processing

Serialization in the sending system

We can use the RBDSER01 report to generate IDocs for the message types of a serialization group. This creates the relevant IDocs from change pointers. We can use the RBDSER02 report to send the IDocs. If required, we can use this report to create the control message SERDAT, which is to trigger processing in the receiving system. If we only want to create a control message, use the RBDSER03 report.

Serialization Using Message Types

Customizing ALE IMG (transaction SALE)/ Model and Implement Business Processes/Configure Master Data Distribution/Set-Up Data for Sending and Receiving Serialization/Serialization by Message Type

Serialization at IDoc Level

IDocs generated by BAPI interfaces cannot be serialized at IDoc level because the function module for inbound processing does not use the ALE Serialization API.

ALE provides two function modules to serialize IDocs which the posting function module has to invoke:

IDOC_SERIALIZATION_CHECK checks the time stamps in the serialization field of the IDoc header.
IDOC_SERIAL_POST updates the serialization table.

SAP recommends that we regularly schedule program RBDSRCLR to clean up table BDSER (old time stamp).

Relationships

Tables: IDOCREL, SRRELROLES

How to add relationship for IDoc

DATA: lv_doc_cat TYPE vbtyp,
        lv_objtype TYPE nast-objtype,
        lv_logsys TYPE tbdls-logsys.

  DATA: ls_obj_rolea TYPE borident,
        ls_obj_roleb TYPE borident.

  SELECT SINGLE vbtyp INTO lv_doc_cat FROM vbak WHERE vbeln = iv_sales_document.

  CALL FUNCTION 'SD_OBJECT_TYPE_DETERMINE'
    EXPORTING
      i_document_type   = lv_doc_cat
    IMPORTING
      e_business_object = lv_objtype
    EXCEPTIONS
      OTHERS            = 1.

  CALL FUNCTION 'OWN_LOGICAL_SYSTEM_GET'
    IMPORTING
      own_logical_system             = lv_logsys
    EXCEPTIONS
      own_logical_system_not_defined = 1
      OTHERS                         = 2.

  CHECK sy-subrc = 0.

  ls_obj_rolea-objkey = iv_idoc_number.
  ls_obj_rolea-objtype = 'IDOC'.
  ls_obj_rolea-logsys = lv_logsys.

  ls_obj_roleb-objkey = iv_sales_document.
  ls_obj_roleb-objtype = lv_objtype.
  ls_obj_roleb-logsys = lv_logsys.

  CALL FUNCTION 'BINARY_RELATION_CREATE'
    EXPORTING
      obj_rolea      = ls_obj_rolea
      obj_roleb      = ls_obj_roleb
      relationtype   = 'IDC1'
    EXCEPTIONS
      no_model       = 1
      internal_error = 2
      unknown        = 3
      OTHERS         = 4.
  IF sy-subrc EQ 0.
    CALL FUNCTION 'BAPI_TRANSACTION_COMMIT'.
  ELSE.
    CALL FUNCTION 'BAPI_TRANSACTION_ROLLBACK'.
  ENDIF.

References

752194 - Serialization of IDoc processing

There are various procedures for processing IDocs in a serialized way. Depending on your respective requirements, one of the proposed methods may be more suitable, or may not be suitable at all for the required purpose.

Note that the IDoc number is not suitable as a serialization characteristic. The number range server assigns the numbers in ranges to the individual application servers. Therefore, in systems that have several application servers, it is very likely that IDocs that have lower numbers will be saved in the system later than IDocs that have higher numbers. Likewise, certain numbers may not be assigned because of number buffering.

Serialization in the inbound:

This type of serialization is available as the default. The prerequisite for it is that inbound processing is set to "Trigger by background program" and that you use the RBDAPP01 report to process the IDocs.

The RBDAPP01 report sorts the IDocs that are to be processed according to the following criteria before processing them:

1. Partner number of the sender

2. Partner type of the sender

3. Partner function of the sender

4. Message type

5. Logical message variant

6. Logical message function

7. Test indicator

8. Serialization field

This type of serialization does not require any Customizing settings or technical adjustments to the program.

Serialization using a time stamp:

You use this type of serialization only if a number of changes occur for an application document and if maintenance may lead to business inconsistencies. The system rejects changes that have an earlier time stamp, and this means that the system only processes the most current change.

The system uses the value in the SERIAL field from the control record of the IDoc as a time stamp. For each scenario with this serialization, the system writes an entry in the BDSER table for each document (depending on the sending and receiving systems). You should therefore use the RBDSRCLR report to clean up this table regularly.

How does time stamp serialization work?

In the sending system, the system generally fills the SERIAL field for each IDoc in the control record of an IDoc with the current time stamp.

In the inbound function module, you must call the IDOC_SERIALIZATION_CHECK function module to check whether there is already an entry in the BDSER table.

If this is the case, the system has to check whether the time stamp of the IDoc is older than the entry from the BDSER table. If it is, the processing of the IDoc terminates with the error status '51'.

The status information and serialization information are returned to the ALE layer, which updates the BDSER table and saves the status of the IDoc.

Checking time stamp serialization:

In transaction BD57, you can check whether an ALE object type is assigned to the message type as a serialization object.

In BD59, you can check whether the serialization object is assigned to a segment field.

You can use the where-used list of the IDOC_SERIALIZATION_CHECK function module to determine the modules that allow time stamp serialization. If yo do not call this function module, then there is no time stamp serialization for the object or scenario that is being checked.

Serialization using message types (serialization groups)

Serialization using serialization groups is used if several different message types are dependent on each other. Therefore, for example, you can assign classes to material masters in logistics. However, it does not make sense to generate this assignment if the definitions of the material masters and classes are not up-to-date. For this reason, the message types involved (MATMAS, CLSMAS and CLFMAS) are grouped together in a serialization group, in which the processing sequence is defined.

How does serialization using groups work?

Serialization in the sending system

In the sending system, check whether there is a serialization group in which the required messages are available in the necessary sequence (transaction BD44).

In the distribution model, maintain a distribution of the SERDAT message type for all target systems to which you want to distribute the messages.

The outbound partner profile in the sending system should be set to 'Collect IDocs' for the message types in question. For the SERDAT message type, the output mode should be set to 'Transfer IDoc immediately'.

You can use the RBDSER01 report to generate IDocs for the message types of a serialization group. This creates the relevant IDocs from change pointers. You can use the RBDSER02 report to send the IDocs. If required, you can use this report to create the control message SERDAT, which is to trigger processing in the receiving system. If you only want to create a control message, use the RBDSER03 report.

Serialization in the target system

Check whether the required serialization group also exists in the target system.

Maintain the inbound settings for all message types of the serialization group. (Transaction SALE: Modeling and Implementing Business Processes --> Master Data Distribution --> Serialization for Sending and Receiving Data --> Serialization using Message Types).

If you leave the field 'Object/Process' empty, the IDocs are transferred separately to the inbound function module of the application, which can result in long runtimes.

If you do not select the field 'Parallel', the inbound function module is called in the current work process. The IDocs are processed one after the other.

If you select the field 'Parallel', parallel processing is used for each package. Since this only concerns the processing of IDocs for a message type, serialization is not compromised.

You must set the inbound partner profile for all message types involved to "Trigger by background program".

Set "Trigger immediately" and the process code SERD for the SERDAT message type. In any case, the inbound processing of SERDAT must trigger the processing of further IDocs indirectly.

Make sure that no jobs that also select the serialized message types are scheduled for the RBDAPP01 and RBDSER04 reports.

Alternatively, when you start inbound processing with the SERDAT message type, you can also use the RBDSER04 report to transfer IDocs to the application.

Specify a variant and a serialization group.

Restrict the logical sending systems where necessary.

If possible, schedule the program for a time when the system workload is low. The settings (TBD41) are taken into account in processing. If you have set parallel processing, processing is executed in several dialog work processes.

Object channel serialization

Object channel serialization ensures that documents are processed in the receiving system in exactly the same sequence in which they were created in the sending system. Numbering makes it possible to detect gaps in the received IDocs and to delay processing until the missing documents have arrived. Waiting documents are given the status value 66 (IDoc is waiting for predecessor IDoc).

The basic requirement is that the sending system ensures that each serialization information is sent only once. If the same serialization information is sent several times, this can have unforeseeable consequences.

Use transaction BD105 to find supported business objects and transaction BD104 to find the assignment to message types.

When the master IDoc is created, you call a certain function module (ALE_SERIAL_KEY2CHANNEL) to create a channel number. This maps a document characteristic key value to a four digit number.

The application transfers this channel number with the master IDoc and the control record to the ALE layer.

The ALE layer uses the Customizing settings to determine the BOR object type that is assigned to the message type.

The next sequence number is determined using this data (BOR type, channel number) and written in the serialization field of the control record.

In inbound processing, the system checks that the number determined is exactly one higher than the value stored in the BDRGIN table. If this is the case, the IDoc is transferred to the application for processing. If not, the IDoc is given the status 66. Prior to processing, the RBDAPP01 report sorts all IDocs for which object channel serialization is activated by BOR object type and channel number.

As soon as the IDoc has been processed successfully, the counter in BDRGIN is updated.

Serialization using qRFC (as of 6.40)

As of Release 6.40, ALE can use qRFC. This method makes it possible to process all LUWs that were sent to a destination in the sequence in which they were created.

In the ALE environment, you can only use qRFC if the sending and receiving systems support this method.

To activate qRFC, you simply have to activate "queue processing" for the message types in question in the partner profiles (transaction: WE20). In addition to this, you must define the rule that determines the queue name. The rules are defined in transaction WE85 and refer to a function module that returns the name of the queue when it is called.

You can use transactions WEINBQUEUE and WEOUTQUEUE to control the individual queues.

Many customers requested that the qRFC function is also implemented in Basis Release 6.20. This is delivered with the Support Package that is mentioned in Note 1004454.

Note:

Also note the following:

All the serialization procedures introduced in this note have advantages and disadvantages that make them suitable for a certain purpose. It is also possible that certain procedures will be excluded for certain scenarios because of the necessary system requirements.

There are also differences in the "resolution" of the serialization. If you use a resolution of one second as a time stamp, nothing that is sent using the time stamp is distinguishable. It is also possible that the time is determined differently on different application servers and, to a certain extent, it also depends on which CPU of a multi-processor system is used.

You must also clarify what is to happen if (for example) an IDoc cannot be processed because of an error, or a predecessor could not be processed.

One difference is whether IDocs are transferred to processing in a serialized way or whether they are processed in a serialized way. In time stamp serialization, the system generally rejects "previous" IDocs. In object channel serialization, on the other hand, IDocs are processed in a serialized way, which means that the processing of the predecessor must be completed successfully.

817939 - IDoc: IDocs remain in status 75

Check whether queuing is necessary. The sending system determines whether IDocs are sent in a queue. Make sure that sufficient resources are available.
Use transaction OYEA to specify a server group for inbound processing.
You can use transaction WEINBQUEUE to display and process IDocs in a queue. If a new IDoc is received for the same queue name, these IDocs are processed automatically.
The reports RSEINBQUEUE_PARTNER and RSEINBQUEUE also trigger inbound processing for IDocs that have the status 75 and can be scheduled as jobs (transaction SM37).
To avoid lock conflicts, use the Note Assistant to implement the correction instructions, or import the relevant Support Package.

1862256 - Inbound queues (SMQ2) stay in status RETRY

Alternatively, the scheduler can also be triggered manually from SMQR, in this case, the scheduler will be triggered and will be running with the user that ran the SMQR and performed the manual activation of the scheduler

Assign S_BTCH_JOB authorization to all users that can create queue entries.

1824405 - Inbound queue scheduler : customize Scheduler Monitoring

The inbound queue scheduling process works at 2 levels:

Main scheduler: Spawns one queue scheduler each for each specific queue name and rolls out (waiting for at least one queue scheduler to return, checking all of them frequently).
Queue scheduler: Executes queue entries in a given queue name. Returns after all the queue entries are processed (or) the MAXRUNTIME set for the queue name is passed.

So at any point in time, there is a maximum of 1 main scheduler running in the system and a maximum of 'n' queue schedulers (depending on the number of DIA workprocesses) in the system where 'n' is the number of queues in transaction SMQ2. The main scheduler is just a controlling process and it does not occupy CPU most of the time.

The main scheduler employs an internal monitoring mechanism that keeps track of the time for which different queue schedulers have been running. If any of them cross a threshold value of 1800 seconds (hardcoded) or the custom scheduler monitoring value set for a queue name, then it does not keep track of this queue anymore and marks this queue as excluded. This is to make sure that the main scheduler can exit whenever it processed all the other queues successfully and need not wait for this particular queue which never seems to finish execution. There is also a system log (after note 1745298) entry with the text QINEXCLUDE which identifies the excluded queue name. Such a queue will be triggered again for execution only with a fresh start of the main scheduler (if that queue is not in RUNNING status already).

This in turn leads to an undesirable behavior when the main scheduler always has other queues to process and never exits execution.

If the main scheduler never exits execution, the excluded queues never get a chance to be triggered again.

2556837 - How to manually restart Q/T RFC scheduler SMQS/SMQR

Either the Inbound Scheduler (SMQS) or Outbound Scheduler (SMQR) mechanism is stuck.

The mechanism requires a manual restart to continue proceeding the q/t RFC processing.

Identify which layer is your issue.

If the issue is related to stuck outbound/transactional entries on SMQ1/SM58, then perform the restart procedure only on QOUT Scheduler (SMQS).
If the issue is related to stuck inbound entries on SMQ2, then perform the restart procedure only on QIN Scheduler (SMQR).

Scheduler restart procedure:

The steps below are the same for both SMQS or SQMR. The scheduler has different mechanisms for inbound/outbound processing, so make sure to restarting the specific mechanism based in your layer.

Access the respective scheduler based in the layer of the issue.

Take note about which destinations/queues are already registered before changing anything.

The column Type values are (R = Registered / U = Unregistered)

IMAGE1.png

Example where two destinations are registered.

De-register all destinations/queues;

deregister.png

Wait for the scheduler to change the status to "Inactive";

Inactive Scheduler.png

Select the "Unregistered" queues/destination (based on step2 list) and Re-register it, clicking on button "Registration".

registration_DP.png

If the mechanism was stuck due to an overload due a specific queue, a good practice is Registering the most important destination/queues for the business perspective first, then after some time Re-register the suspicious destination/queue.

Analysis of the FM involved

Getting application details for qRFCs (SMQ1/SMQ2)

Open the transaction SMQ1/SMQ2 and filter by preferred queues.
Select the queue then double click the Queue Name to open Queue details.
On queue details, double click the Queue name again to open LUW details.
On LUW details, identify the Function Module name.
The affected application is related to this Function Module component.

Getting application details for tRFC (SM58)

Open SM58 using the preferred filters.
Identify the Function Module information for the affected entry.
The affected application is related to this Function Module component.

Finding the component of responsible application based on Function Module

To find the component of the application team responsible for this Function Module, use the direct link to Navigate on Guided Answer or follow the Guided Answers described in this KBA 2474005:
- Select option: 2) Check the component of affected objects.
- Select option: Function Module / BAPI.

1804549 - Restarting Inbound SYSFAIL, RETRY, CPIERR and RUNNING Queues

You should use report RSQIWKEX.

Report RSQIWKEX resets the status of all queues (CPICERR, RETRY, SYSFAIL and RUNNING) and restarts the Inbound Scheduler.

Search This Blog

Toolkit for SAP consultants