 India SAP CoE, Slide 1 BI1003 – Data Extraction – v2.0.

 India SAP CoE, Slide 1 BI1003 – Data Extraction – v2.0

 India SAP CoE, Slide 2 Data Extraction 1 PrepareMe 2 TellMe 3 ShowMe 4 LetMe 5 HelpMe

 India SAP CoE, Slide 4 PrepareMe In this course, we will cover 3 main sub-topics : - Purpose - Purpose - Use - Use - Challenges - Challenges

 India SAP CoE, Slide 5 Purpose This course demonstrates the various data extraction methods.This course demonstrates the various data extraction methods. Extraction Methods coveredExtraction Methods covered –Business Content Extraction from SAP R/3 –LIS & LO Extraction from SAP R/3 –CO-PA and FI-SL Extraction –Generic Extractors –Flat File Extraction –UD Connect, DB Connect, SOAP Based and Open Hub Extraction DataSource EnhancementDataSource Enhancement

 India SAP CoE, Slide 6 Use In many organizations, data is fragmented and spread across many databases and applications.In many organizations, data is fragmented and spread across many databases and applications. To be useful, data must be integrated, standardized, synchronized, and enriched – typically through ETL (extraction, transformation, and loading) processes.To be useful, data must be integrated, standardized, synchronized, and enriched – typically through ETL (extraction, transformation, and loading) processes. SAP BI provides a broad set of ETL capabilities that support data extraction.SAP BI provides a broad set of ETL capabilities that support data extraction. With the open interfaces of SAP BI, data can be loaded from virtually any source and can handle the huge amounts of transactional data typical of the enterprise landscape.With the open interfaces of SAP BI, data can be loaded from virtually any source and can handle the huge amounts of transactional data typical of the enterprise landscape. Thus extraction deals with extracting the data into the BW system for analysing and reporting.Thus extraction deals with extracting the data into the BW system for analysing and reporting.

 India SAP CoE, Slide 7 Challenges To extract large volumes of data from SAP R/3 by defining delta managementTo extract large volumes of data from SAP R/3 by defining delta management Modifying the already available Business content and satisfying user needsModifying the already available Business content and satisfying user needs Integrating data from non SAP systems into BWIntegrating data from non SAP systems into BW

 India SAP CoE, Slide 8 Data Extraction 1 PrepareMe 2 TellMe 3 ShowMe 4 LetMe 5 HelpMe ShowMe

 India SAP CoE, Slide 9 Data Extraction : Overview BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 10 Update Rules Transfer Rules Update Rules Transfer Rules ODS Process Flow PSA DataSource Update Rules InfoCube Delivery Order Delivery OLTP System Data extract Order-Deliv.

 India SAP CoE, Slide 11 Process Flow The data flows from the OLTP system to the PSA tables without any transformation.The data flows from the OLTP system to the PSA tables without any transformation. If any transformation or filtration is required in the data it can be done in the update rules and transfer rules.If any transformation or filtration is required in the data it can be done in the update rules and transfer rules. Then the data is loaded in the data targets like ODS. An ODS object acts as a storage location for consolidated and cleaned-up transaction data.Then the data is loaded in the data targets like ODS. An ODS object acts as a storage location for consolidated and cleaned-up transaction data. The data from multiple ODS is then loaded into InfoCube and thus available to generate reports.The data from multiple ODS is then loaded into InfoCube and thus available to generate reports. In this example, Order and Delivery Data are extracted separately, and populate separate PSA and ODS tables. The data is then merged within the ODS to form a new ODS object storing combined Order Delivery data. The data then updates the InfoCube with Order Delivery data.In this example, Order and Delivery Data are extracted separately, and populate separate PSA and ODS tables. The data is then merged within the ODS to form a new ODS object storing combined Order Delivery data. The data then updates the InfoCube with Order Delivery data.

 India SAP CoE, Slide 12 Multiple Source Systems

 India SAP CoE, Slide 13 Types of Extractors FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 14 Types of Extractors Extractors mainly are of two types:Extractors mainly are of two types: –Application Specific –Cross Application Application Specific Extractors consist ofApplication Specific Extractors consist of –BI Content Extractors (LO Cockpit) –Customer Generated Extractors (CO-PA, FI-SL and LIS) Cross Application Extractors consist of generic extractors (based on Database table, Infoset and Function module)Cross Application Extractors consist of generic extractors (based on Database table, Infoset and Function module)

 India SAP CoE, Slide 15 Business Content Extraction FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM Business Content Extractors: Application Specific Extractors that are activated when business content datasources are transferred

 India SAP CoE, Slide 16 Business Content Extractors SAP provides extractors for almost all applications. They are provided as an add on with the Business Content.SAP provides extractors for almost all applications. They are provided as an add on with the Business Content. For each application like FI,CO and HR there are specific tables in the OLTP system.For each application like FI,CO and HR there are specific tables in the OLTP system. BI Content extractors consist of extract structures which are based upon these tables and thus can be used to fetch the data in BW.BI Content extractors consist of extract structures which are based upon these tables and thus can be used to fetch the data in BW. Hence when any company implements application like FI, CO it can directly use the already available BI Content Extractors without the need to create their own.Hence when any company implements application like FI, CO it can directly use the already available BI Content Extractors without the need to create their own. SAP delivers these objects in Delivery version (D) and we need to convert them in the active version (A).SAP delivers these objects in Delivery version (D) and we need to convert them in the active version (A). Business content DataSources from a Source System are available in BW for transferring data only after you have converted them into the active versions in the source system and replicated them.Business content DataSources from a Source System are available in BW for transferring data only after you have converted them into the active versions in the source system and replicated them.

 India SAP CoE, Slide 17 Uses of BI Content Extractors Built in ExtractorsBuilt in Extractors High coverage (Applications and Industries)High coverage (Applications and Industries) Available for both Transaction and Master dataAvailable for both Transaction and Master data Ready to useReady to use Reduce implementation Costs and effortsReduce implementation Costs and efforts Delta capableDelta capable

 India SAP CoE, Slide 18 LO Data Extraction : Overview FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 19 LO Cockpit Logistics Customizing Cockpit provides a simplified way to extract logistics data and transfer it to SAP Business Information Warehouse.Logistics Customizing Cockpit provides a simplified way to extract logistics data and transfer it to SAP Business Information Warehouse.

 India SAP CoE, Slide 20 LO Data Extraction : Data Flow As shown in the figure the document data for various applications like Customer order, delivery and billing is first transferred to various Communication Structures.As shown in the figure the document data for various applications like Customer order, delivery and billing is first transferred to various Communication Structures. These communication structures are used to form various extract structures in R/3 which consist of various DataSources like 2LIS_11_VAHDR, 2LIS_11_VAITM etc.These communication structures are used to form various extract structures in R/3 which consist of various DataSources like 2LIS_11_VAHDR, 2LIS_11_VAITM etc. These DataSources then can be replicated to BW and assigned to Infosources.These DataSources then can be replicated to BW and assigned to Infosources. Then by creating transfer rules and update rules and defining the transformation the data is loaded into various data targets like InfoCube and ODS, thus available for reporting.Then by creating transfer rules and update rules and defining the transformation the data is loaded into various data targets like InfoCube and ODS, thus available for reporting.

 India SAP CoE, Slide 21 V1 and V2 Updates V1 Update – Synchronous UpdateV1 Update – Synchronous Update –If you create/change a purchase order (me21n/me22n), when you press 'SAVE' and see a success message (PO.... changed..), the update to underlying tables EKKO/EKPO has happened (before you saw the message). This update was executed in the V1 work process. V2 Update – Asynchronous UpdateV2 Update – Asynchronous Update –If you create/change a purchase order (me21n/me22n), when you press 'SAVE' it takes few seconds to update to underlying tables EKKO/EKPO depending on system load. This update was executed in the V2 work process. V3 Update has now replaced V1 and V2 updates.V3 Update has now replaced V1 and V2 updates.

 India SAP CoE, Slide 22 V3 Update Modes in LO Cockpit There are four Update Modes in LO ExtractionThere are four Update Modes in LO Extraction –Serialized V3 Update –Direct Delta –Queued Delta –Un-serialized V3 Update

 India SAP CoE, Slide 23 Serialized V3 Update Transaction tables Application specific customer exits Update Tables BW Delta queue Staging Engine Periodic Job Delta Load SM 13 RSA7 R3 systemBW System

 India SAP CoE, Slide 24 Serialized V3 Update Transaction data is collected in the R/3 update tableTransaction data is collected in the R/3 update table Data in the update tables is transferred through a periodic update process to BW Delta queueData in the update tables is transferred through a periodic update process to BW Delta queue Delta loads from BW retrieve the data from this BW Delta queueDelta loads from BW retrieve the data from this BW Delta queue Transaction postings lead to:Transaction postings lead to: –Records in transaction tables and in update tables –A periodically scheduled job transfers these postings into the BW delta queue –This BW Delta queue is read when a delta load is executed.

 India SAP CoE, Slide 25 Direct Delta Transaction tables Application specific customer exits BW Delta queue Staging Engine Delta Load R3 systemBW System

 India SAP CoE, Slide 26 Direct Delta Each document posting is directly transferred into the BW delta queueEach document posting is directly transferred into the BW delta queue Each document posting with delta extraction leads to exactly one LUW in the respective BW delta queuesEach document posting with delta extraction leads to exactly one LUW in the respective BW delta queues Transaction postings lead to:Transaction postings lead to: –Records in transaction tables and in update tables –A periodically scheduled job transfers these postings into the BW delta queue –This BW Delta queue is read when a delta load is executed.

 India SAP CoE, Slide 27 Queued Delta Transaction tables Application specific customer exits Extraction queue BW Delta queue Staging Engine Periodic Job Delta Load LBWQ RSA7 R3 systemBW System

 India SAP CoE, Slide 28 Queued Delta Extraction data is collected for the affected application in an extraction queueExtraction data is collected for the affected application in an extraction queue Collective run as usual for transferring data into the BW delta queueCollective run as usual for transferring data into the BW delta queue Transaction postings lead to:Transaction postings lead to: –Records in transaction tables and in extraction queue –A periodically scheduled job transfers these postings into the BW delta queue –This BW Delta queue is read when a delta load is executed.

 India SAP CoE, Slide 29 Un-Serialized V3 Update Transaction tables Application specific customer exits Update Tables BW Delta queue Staging Engine Periodic Job Delta Load SM 13 RSA7 R3 systemBW System

 India SAP CoE, Slide 30 Un-Serialized V3 Update Extraction data for written as before into the update tables with a V3 update moduleExtraction data for written as before into the update tables with a V3 update module V3 collective run transfers the data to BW Delta queueV3 collective run transfers the data to BW Delta queue In contrast to serialized V3, the data in the updating collective run is without regard to sequence from the update tablesIn contrast to serialized V3, the data in the updating collective run is without regard to sequence from the update tables Transaction postings lead to:Transaction postings lead to: –Records in transaction tables and in update table –A periodically scheduled job transfers these postings into the BW delta queue –This BW Delta queue is read when a delta load is executed.

 India SAP CoE, Slide 31 New LO Data Extraction : Why ?

 India SAP CoE, Slide 32 New LO Data Extraction : Why ? Performance and Data VolumePerformance and Data Volume – –Detailed extraction: You can deactivate the extraction of, for example, scheduling data ('thin extractors' lead to small upload volumes) – –Document changes: only BW-relevant data changes are updated (smaller upload volumes) – –LIS tables are not updated: Reduced data volumes due to removed redundant data storage – –Update with batch processes: No overload of everyday work'

 India SAP CoE, Slide 33 New LO Data Extraction : Why ? Simple Handling – –BW Customizing Cockpit: Central, single, maintenance tool for Logistics applications – –No LIS functionality: No knowledge of LIS Customizing update settings, etc, required – –Function enhancement: Creating simply and without modifications Others – –Delta tables (SnnnBIW1/-2): duplicated update, duplicated data storage

 India SAP CoE, Slide 34 Customer Generated Extractors FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 35 Customer Generated Extractors In response to demands from the customers, SAP has designed the reporting facility in certain parts of the R/3 system in such a way that it can be easily customized to meet customer’s requirements.In response to demands from the customers, SAP has designed the reporting facility in certain parts of the R/3 system in such a way that it can be easily customized to meet customer’s requirements. The Business Information Warehouse must be able to process this data, with the result that it must be possible to define generic extractors in certain applications. The Business Information Warehouse must be able to process this data, with the result that it must be possible to define generic extractors in certain applications. These applications are the These applications are the –Logistics Information System –Profitability analysis –Special Ledger

 India SAP CoE, Slide 36 LIS Extraction FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 37 LIS Data Extraction

 India SAP CoE, Slide 38 LIS Data Extraction The LIS data flow can be described as follows: Operative transaction data (sales orders, billing) is stored in application data tables (for example VBAK/VBAP for sales orders) The information is transferred from an operative document to the Communication Structures (for example MCVBAK,MCVBAP) to update the data in the LIS). These R/3 Communication perform the same function as the BW Communication Structures, but they fill LIS InfoStructures in R/3 with data and not cubes in BW. The update rules write the information from the Communication Structure to the corresponding Information Structures.

 India SAP CoE, Slide 39 LIS Data Extraction Although the term InfoStructure is used in R/3, the InfoStructure is a real transparent table and not the structure which contains data only at run time.Although the term InfoStructure is used in R/3, the InfoStructure is a real transparent table and not the structure which contains data only at run time. LIS has many reporting tools like ABC analysis, correlation, graphical displays but the limitations of the schema with only one table makes reporting slower than it would be in BW.LIS has many reporting tools like ABC analysis, correlation, graphical displays but the limitations of the schema with only one table makes reporting slower than it would be in BW. Thus in the LIS Extractor the data is transferred from the LIS InfoStructure in BW.Thus in the LIS Extractor the data is transferred from the LIS InfoStructure in BW. LIS extractors are now obsolete and replaced by LO Cockpit.LIS extractors are now obsolete and replaced by LO Cockpit.

 India SAP CoE, Slide 40 FI-SL Extractors FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 41 FI-SL Extractors FI-SL is an application in which data (planned and actual) from different levels of OLTP applications is combined to measure business performance.FI-SL is an application in which data (planned and actual) from different levels of OLTP applications is combined to measure business performance. FI-SL includes planning functions and reporting tools. FI-SL reporting in SAP R/3 is however restricted by the following :FI-SL includes planning functions and reporting tools. FI-SL reporting in SAP R/3 is however restricted by the following : –Cross application reporting is not diverse –The OLTP system is optimized for transaction processing and a high reporting workload would have a negative impact on the overall performance of the system The solution to the above limitation is FI-SL Reporting in BW.The solution to the above limitation is FI-SL Reporting in BW.

 India SAP CoE, Slide 42 Uses of FI-SL Extractors In FI-SL one is able to use alternate chart of accounts (operative, group specific, country specific charts of accounts)In FI-SL one is able to use alternate chart of accounts (operative, group specific, country specific charts of accounts) Adjustment postings (direct data entry) can be made in the FI-SL systemAdjustment postings (direct data entry) can be made in the FI-SL system Various fiscal year variants enables one to create a weekly or monthly reportVarious fiscal year variants enables one to create a weekly or monthly report Validations or substitutions allow one to check or modify the data whenever it enters the FI-SL system.Validations or substitutions allow one to check or modify the data whenever it enters the FI-SL system. Up to three currencies and two quantities can be run in parallel in FI-SLUp to three currencies and two quantities can be run in parallel in FI-SL Selective data retrieval - the assignment of transactions to particular company code or ledgers determines which ledgers are to be updated.Selective data retrieval - the assignment of transactions to particular company code or ledgers determines which ledgers are to be updated.

 India SAP CoE, Slide 43 FI-SL Data Flow

 India SAP CoE, Slide 44 CO-PA Extractors FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 45 CO-PA Extractors Profitability Analysis (PA) is an integrated component in the SAP R/3 system.Profitability Analysis (PA) is an integrated component in the SAP R/3 system. All of the data related to profitability from the other SAP R/3 applications is mapped in CO-PA in accordance with the corresponding business transactions.All of the data related to profitability from the other SAP R/3 applications is mapped in CO-PA in accordance with the corresponding business transactions. This allows you to transfer into CO-PA billing document data from SD, cost object costs from CO-PC, and overall costs from overall cost controlling.This allows you to transfer into CO-PA billing document data from SD, cost object costs from CO-PC, and overall costs from overall cost controlling. CO-PA collects all of the OLTP data for calculating contribution margins (sales, cost of sales, overhead costs)CO-PA collects all of the OLTP data for calculating contribution margins (sales, cost of sales, overhead costs) CO-PA also has powerful reporting tools and planning functionsCO-PA also has powerful reporting tools and planning functions

 India SAP CoE, Slide 46 CO-PA Extractors

 India SAP CoE, Slide 47 CO-PA Extractors During billing SD, revenues and payments are transferred to profitability segments in Profitability Analysis. At the same time, sales quantities are valuated using the standard cost of goods manufactured, as specified in the cost component split from CO-PC.During billing SD, revenues and payments are transferred to profitability segments in Profitability Analysis. At the same time, sales quantities are valuated using the standard cost of goods manufactured, as specified in the cost component split from CO-PC. One of the typical questions that can be answered with CO-PA module is what are the top products and customers in our different divisions.One of the typical questions that can be answered with CO-PA module is what are the top products and customers in our different divisions. The wide variety of analysis and planning functions in CO-PA allow you to plan, monitor and control the success of your company in terms of product-oriented, customer-oriented and organizational dimensions of multidimensional profitability segments.The wide variety of analysis and planning functions in CO-PA allow you to plan, monitor and control the success of your company in terms of product-oriented, customer-oriented and organizational dimensions of multidimensional profitability segments.

 India SAP CoE, Slide 48 Generic Extractors FI SAP CRM Types of Extractors Application SpecificCross Application BW Content Extractors Customer Generated Extractors Generic Extractors LO COCKPIT HRCO LIS FISL COPA DB View Infoset Function Module Application Specific DB Tables Various SAP DB tables Application Specific DB Tables FI SAP CRM

 India SAP CoE, Slide 49 Generic Extractors Generic Extractors are Cross Application Extractors used in scenarios where the other type of extractors are unavailable.Generic Extractors are Cross Application Extractors used in scenarios where the other type of extractors are unavailable. Generic extractors are of three types:Generic extractors are of three types: –Based on Database view/Table –Based on InfoSet –Based on Function Module

 India SAP CoE, Slide 50 Generic Extractors When should you use generic extractors?When should you use generic extractors? –Business Content does not contain a DataSource for your application. –The application does not feature its own generic delta extraction method extraction method –You are using your own programs in SAP R/3 to populate the tables The tools for generic data extraction can be used to generate an extractor for all applications.The tools for generic data extraction can be used to generate an extractor for all applications.

 India SAP CoE, Slide 51 Flat file Extraction BW provides facility to load data from flat files (CSV or ASCII files).BW provides facility to load data from flat files (CSV or ASCII files). It supports the following types of data :It supports the following types of data : –Transaction data –Master data, either directly or flexibly AttributesAttributes TextText HierarchiesHierarchies Flat file can be stored either on a local system or Application Server.Flat file can be stored either on a local system or Application Server. But from performance point of view we store the file on the applications server and then load it into BW.But from performance point of view we store the file on the applications server and then load it into BW.

 India SAP CoE, Slide 52 UD Connect Overview With the help of the SAP Web AS J2EE connectivity, UD With the help of the SAP Web AS J2EE connectivity, UD Connect (Universal Data Connect) enables reporting and Connect (Universal Data Connect) enables reporting and analysis of both SAP and non-SAP data. analysis of both SAP and non-SAP data. Using UD Connect, you can access just about all relational Using UD Connect, you can access just about all relational and multi-dimensional data sources. and multi-dimensional data sources. UD Connect transfers the data as flat data. Multi- UD Connect transfers the data as flat data. Multi- dimensional data is converted to a flat format when UD dimensional data is converted to a flat format when UD Connect is used. Connect is used.

 India SAP CoE, Slide 53 UD Connect Overview For the connection to DataSources (in BW), UD Connect uses the J2EE Connector Architecture.For the connection to DataSources (in BW), UD Connect uses the J2EE Connector Architecture. BI Java Connectors that are available for various drivers, protocols and providers as resource adapters.BI Java Connectors that are available for various drivers, protocols and providers as resource adapters. – BI JDBC Connector – BI ODBO Connector – BI SAP Query Connector – BI XMLA Connector

 India SAP CoE, Slide 54 DB Connect Overview Unix Foreign Application ORACLE DataBase Purchasing SAP BW MS SQL DataBase NT DB-Connect

 India SAP CoE, Slide 55 DB Connect Overview A purchasing application runs on a legacy system based on an ORACLE database. To analyze the data from the purchasing application, the data needs to be loaded into the BW System (possibly installed on a different database, e.g. MS SQL-Server). DB-Connect can be used to connect the DBMS of of the purchasing application and extract data from tables or views.

 India SAP CoE, Slide 56 Data Transfer with DB Connect

 India SAP CoE, Slide 57 Data Transfer with DB Connect By default, when a BW application server is started by the SAP kernel, the system opens up a connection to the database on which the SAP system runs. All SQL commands (irrespective of whether they are Open or Native SQL) that are submitted by the SAP kernel or by ABAP programs, relate automatically to this default connection, meaning that they run in the context of the database transaction that is active in this connection.

 India SAP CoE, Slide 58 Data Transfer with DB Connect Connection data, such as database user name, user password, or database name are taken either from the profile parameters or from the corresponding environment variables (this is database-specific). Thus DB Connect can be used to open up other database connections in addition to the default connection and use these connections to transfer data from tables or views into a BW system.

 India SAP CoE, Slide 59 SOAP-Based Transfer of Data

 India SAP CoE, Slide 60 SOAP-Based Transfer of Data As a rule, data transfer in BW takes place using a data request that is sent from BW to the Source System (pull from the scheduler). You can also send the data to SAP BW using external control. This is a data push in the SAP BW. Data push is possible for multiple scenarios: – –Transferring Data Using the SOAP Service SAP Web AS – –Transferring Data Using Web Service – –Transferring Data Using SAP XI In all three scenarios, data transfer takes place via transfer mechanisms that are sufficient for Simple Object Access Protocol (SOAP) and are XML based.

 India SAP CoE, Slide 61 Open Hub Service

 India SAP CoE, Slide 62 Open Hub Service The open hub service enables you to distribute data from an SAP BW system into external data marts, analytical applications, and other applications. With this, you can ensure controlled distribution using several systems. The central object for the export of data is the InfoSpoke. Using this, you can define the object from which the data comes and into which target it is transferred. Through the open hub service, SAP BW becomes a hub of an enterprise data warehouse. The distribution of data becomes clear through central monitoring from the distribution status in the BW system.

 India SAP CoE, Slide 63 Datasource Enhancement Need for DataSource enhancement Need for DataSource enhancement – –When we require an additional field from a database table and it is not directly available from the datasource, then we can append the field in the extract structure of the datasource. – –Then the logic to populate that field can be written in the customer exit.

 India SAP CoE, Slide 64 1 PrepareMe 2 TellMe 3 ShowMe 4 LetMe 5 HelpMe Data Extraction

 India SAP CoE, Slide 65 BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 66 BI Content Extractors – Steps 1. Activate Datasources 2. Replicate the Datasource in BW System 3. Assign InfoSource and maintain communication structure and transfer rules 4. Maintain InfoCube and Update Rules 5. Create InfoPackage 6. Initialize Delta Process 7. Carry Out Delta Update

 India SAP CoE, Slide 67 Step 1 - Activate DataSources In SAP R/3, go to transaction Code SBIW  Display IMG  Business Content DataSources  Execute Transfer Business Content DataSources

 India SAP CoE, Slide 68 Step 1 - Activate DataSources Select the DataSource from Application Component Hierarchy. Click on the button Transfer DataSources/ Activate the DataSource. Assign to a transport request. Activate datasources

 India SAP CoE, Slide 69 Step 2 - Replicate datasources Tcode RSA1  Modelling  Source Systems  Context Menu  ReplicateDataSources.

 India SAP CoE, Slide 70 Step 3- Assign the InfoSource Maintain the communication structure and Activate the transfer rules.

 India SAP CoE, Slide 71 Step 4 - Maintain InfoCube and Update rules Infocube Update rules

 India SAP CoE, Slide 72 Step 5 - Create InfoPackage Go to InfoSources, Select the InfoSource Right click on the Source System and say create InfoPackage Scheduler screen Monitor

 India SAP CoE, Slide 73 Step 6 - Initialize the delta process Go to InfoPackage and select the update type as initialize delta process. And click on start to schedule the process Monitor Scheduling Initialize the delta process

 India SAP CoE, Slide 74 Initializing the Delta Process (Scheduler) If you want to transfer data in the delta process, the process must be initialized in the first data request. In this process, the selection conditions for connecting delta uploads are set and its complete dataset is loaded into BW from the R/3 System. To do this, you must select the checkbox Initialize delta process for the Info Package under the update parameters of the Scheduler. Once the data is requested message appears after scheduling, Go to Monitor to view the status and details of the data load.

 India SAP CoE, Slide 75 Monitoring the Upload Procedure

 India SAP CoE, Slide 76 Delta Update (Scheduler) Only the data since last upload is requested

 India SAP CoE, Slide 77 Delta Update (Scheduler) For high volume of transaction data, a full update here is mostly only justified for the first time data is transferred or if the statistics data has to be rebuilt following a system failure. Delta update mechanisms that restrict the volume of data to within realistic limits, therefore, are required to implement a performance-oriented, periodical data transfer strategy. – –For e.g. when sales figures are updated every week in the Business Information Warehouse, only the sales document information that has been added or changed in the last week should be extracted.

 India SAP CoE, Slide 78 LO Cockpit BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 79 Logistics Customizing Cockpit The LO Cockpit contains the following functionality, as specified in the procedure sequenced below: – –Maintaining extract structures – –Maintaining DataSources – –Activating the update – –Controlling V3 update

 India SAP CoE, Slide 80 LO Data Extraction (SBIW) LO customizing cockpit lbwe BW Logs (LBWF) Perform set up Application wise Delete set up tables (LBWG)

 India SAP CoE, Slide 81 Logistics Extract Structure – Cockpit (LBWE) Maintain extract structure Set up V3 update Activate datasources

 India SAP CoE, Slide 82 Individual Steps in LO Extraction 1. Maintain Extract Structure 2. Replicate the Datasource in BW System 3. Assign InfoSource and maintain communication structure and transfer rules 4. Maintain InfoCube and Update Rules 8. Initialize Delta Process 9. Carry Out Delta Update 5. Activate Extract Structure 6. Delete setup tables/Perform Extraction 7. Set Up periodic V3 update

 India SAP CoE, Slide 83 Step 1 - Maintaining the Extract Structure Maintain extract structure

 India SAP CoE, Slide 84 Maintaining the Extract Structure The extract structure is filled with the assigned communication structures. You can only use selected fields from the communication structures (SAP internal control fields, for example, are not offered). SAP already delivers extract structures, which you can enhance (by connecting to the communication structure). Every extract structure can be maintained by you and by SAP. After you set up the extract structure, the system automatically generates it. This completes missing fields (their units and characteristics). The extract structure is created hierarchically according to the communication structures. Every communication structure leads to the generation of a substructure that belongs to the actual extract structure.

 India SAP CoE, Slide 85 Maintaining DataSources Maintain datasources

 India SAP CoE, Slide 86 Maintaining DataSources : Procedure There exists a DataSource (for e.g. 2LIS_11_VAITM) for each extract structure that is made available (for example, MC11VA0ITM) in the OLTP System. A maintenance screen is displayed in which you can to assign other properties to the fields of the extract structure – –Selection fields – –Hide fields – –Cancellation fields (field is inverted when canceled (*-1))

 India SAP CoE, Slide 87 Step 2 – Replicating DataSources

 India SAP CoE, Slide 88 Step 3 - Maintaining Communication Structure All the fields in the extract structure are shown with the assigned info objects in the template

 India SAP CoE, Slide 89 Step 4 - Maintaining Transfer Rules

 India SAP CoE, Slide 90 Step 5 - Maintaining InfoCubes and Update Rules

 India SAP CoE, Slide 91 Step 6 - Activating Extract Structures Activate extract structure

 India SAP CoE, Slide 92 Step 7 - Delete set up tables In transaction LBWG enter the application number for which you want to delete the set up tables. For e.g. For 2LIS_11_VAITM enter the number 11 and execute.

 India SAP CoE, Slide 93 Step 8 - Initialization/Simulation (OLI*BW) Give a name to the job Initializing ('build') the dataset for the respective extract structures is to be done before data can be extracted from the R/3 System. The initialization is set up in the OLTP. The restructuring process fills restructuring tables that are read during the initialization. It is recommended to carry this out before, in the simulation mode. The option of checking the extract structure update, without actually updating the data is available. The results are recorded in a detailed simulation log.

 India SAP CoE, Slide 94 Step 8 - Creating Infopackages

 India SAP CoE, Slide 95 Step 9 - Initializing the Delta Process

 India SAP CoE, Slide 96 V3 Update- Settings Set up V3 update Set the start time and print parameters. Schedule batch jobs

 India SAP CoE, Slide 97 V3 Update Settings In Start Date Select immediate scheduling Give the print parameters Schedule the job and check the job overview in SM37

 India SAP CoE, Slide 98 Delta Queue Maintenance Go to transaction RSA7. You can see that the delta queue has been successfully filled.

 India SAP CoE, Slide 99 Step 10 -Delta Update (Scheduler) Only the data since last upload is requested

 India SAP CoE, Slide 100 FI-SL Extractors BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 101 FI-SL extraction Step-by-Step 2. Define or Assign a DataSource for Ledger 3. Replicate the Datasource in BW System 4. Assign InfoSource and maintain communication structure and transfer rules 5. Maintain InfoCube and Update Rules 6. Create InfoPackage 7. Initialize Delta Process 8. Carry Out Delta Update 1. Generate an extract structure for summary table

 India SAP CoE, Slide 102 Step 1 - Setting up an FI-SL DataSource

 India SAP CoE, Slide 103 Go to SBIW  Settings for Application specific data sources  Generate transfer structure for totals table Go to SBIW  Settings for Application specific data sources  Generate transfer structure for totals table Step 1 - Setting up an FI-SL DataSource

 India SAP CoE, Slide 105 Step 2 - Setting up an FI-SL datasource : Go to SBIW  Settings for application specific datasources  Financial accounting  create infosource for Ledger Go to SBIW  Settings for application specific datasources  Financial accounting  create infosource for Ledger

 India SAP CoE, Slide 106 Step 2 - Setting up an FI-SL DataSource The status column shows whether a DataSource is assigned to the ledger already. (Traffic light green) The status column shows whether a DataSource is assigned to the ledger already. (Traffic light green) You select the ledger to which you want to assign the DataSource You select the ledger to which you want to assign the DataSource

 India SAP CoE, Slide 108 FI-SL Extractors After generating the FI-SL DataSource rest of the steps are same as explained earlier.After generating the FI-SL DataSource rest of the steps are same as explained earlier. Replicate the DataSource and then assign it to the infosource.Replicate the DataSource and then assign it to the infosource. Create transfer rules and update rulesCreate transfer rules and update rules Create InfoPackage and initialize the delta processCreate InfoPackage and initialize the delta process

 India SAP CoE, Slide 109 CO-PA Extractors BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 110 CO-PA Extraction (Steps) 2. Replicate the Datasource in BW System 3. Assign InfoSource and maintain communication structure and transfer rules 4. Maintain InfoCube and Update Rules 5. Create InfoPackage 6. Initialize Delta Process 7. Carry Out Delta Update 1. Generate DataSource in OLTP

 India SAP CoE, Slide 111 Step 1 - Generate CO-PA DataSource Go to Transaction KEB0. Define the DataSource for the current client of your SAP R/3 System on the basis of one of the operating concerns. A standard name is always generated. For e.g. 1_CO_PA%CL%ERK with prefix 1_CO_PA is mandatory. Select the operating concern

 India SAP CoE, Slide 112 Step 1 - Generate DataSource

 India SAP CoE, Slide 113 Maintain CO-PA datasource You can then specify which fields are to be applied as the selection for the CO-PA extraction. Providing required fields for selection

 India SAP CoE, Slide 114 CO-PA Extractors After generating the CO-PA DataSource rest of the steps are same as explained earlier.After generating the CO-PA DataSource rest of the steps are same as explained earlier. Replicate the DataSource and then assign it to the infosource.Replicate the DataSource and then assign it to the infosource. Create transfer rules and update rulesCreate transfer rules and update rules Create InfoPackage and initialize the delta processCreate InfoPackage and initialize the delta process

 India SAP CoE, Slide 115 Generic Extractors BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 116 Generic Data Extraction Steps for transaction Data 1. Create DataSources for transaction Data 3. Replicate the Datasource in BW System 4. Assign InfoSource and maintain communication structure and transfer rules 5. Maintain InfoCube and Update Rules 6. Create InfoPackage 7. Initialize Delta Process 8. Carry Out Delta Update 2. Create InfoSet/enhance table

 India SAP CoE, Slide 117 Generic Data Extraction Steps for Master Data 1. Create DataSources for Master Data (attr) 3. Replicate the Datasource in BW System 4. Assign InfoSource and maintain communication structure and transfer rules 5. Create InfoPackage for Master Data 6. Create InfoPackage for texts 7. Schedule InfoPackages 2. Create DataSources for texts

 India SAP CoE, Slide 118 Step1 - Create generic DataSource Go to Transaction RSO2  Here you can select whether you want to create a datasource for transaction data, Attributes or texts. Give a technical name to the datasource and choose create

 India SAP CoE, Slide 119 Create Generic DataSource Create a Generic DataSource. Choose an application component to which the DataSource is to be assigned. Enter the description texts. You can choose whether you want to create the DataSource based on View, Function Module or InfoSet query. Choose Generic delta

 India SAP CoE, Slide 120 Generic Data Extraction Specify the delta specific field and its type. Maintain the settings for generic delta. Specify the safety interval. NOTE: safety interval should be set so that no document is missed even if it was not stored in the DB table when extraction took place. Select delta type : New status for changed records (i.e. after image) This can be used with data target ODS (AIE). Additive delta i.e. aggregated data records (ADD) and then choose save. Only one attribute can be defined as delta attribute

 India SAP CoE, Slide 121 Generic Data Extraction. Now choose save again. This will generate the DataSource. After generating the DataSource you will see the delta update flag selected. Select the fields in the selection which you want to appear in the Infopackage selection.

 India SAP CoE, Slide 122 Generic Data Extraction Transport the DataSource to BW in Tcode RSA6(Post Processing of DataSources). The steps in BW : Replicate the DataSources.Replicate the DataSources. Create necessary Info objectsCreate necessary Info objects Generate Info source in BW, Maintain Communication Structure and Transfer RulesGenerate Info source in BW, Maintain Communication Structure and Transfer Rules Create Info package and schedule it.Create Info package and schedule it.

 India SAP CoE, Slide 123 Flat File Extraction BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 124 Flat File Extraction 1. Identify the data 3. Define fields of InfoSource as InfoObjects 4. Assign InfoSource and maintain communication structure and transfer rules 5. Maintain InfoCube and Update Rules 6. Create InfoPackage 7. Schedule Flat file using extractor 2. Define Source System in AWB

 India SAP CoE, Slide 125 Step 1 – Identify the Data Go to MS Excel. Create a flat file and enter the records to be updated in the file. Save the file as a.CSV file. NOTE: The order of the fields in the flat file should be same as that in the communication structure of the InfoSource.

 India SAP CoE, Slide 126 Step 2 - Create a Source System Step 2 - Create a Source System RSA1 Modelling  Source Systems  Context Menu Create  Select the Source System as File System.

 India SAP CoE, Slide 127 Flat File Extraction After creating the Source System, create the Infoobjects in the BW system.After creating the Source System, create the Infoobjects in the BW system. Create an InfoSource and assign the flat file Source System to it and maintain the transfer rules.Create an InfoSource and assign the flat file Source System to it and maintain the transfer rules. Create and ODS and maintain update rules.Create and ODS and maintain update rules. Now create an infopackage and load the data.Now create an infopackage and load the data.

 India SAP CoE, Slide 128 Hints while loading flat file Data separator File in CSV format Give the name of the CSV file which you have created in the infopackage. Use the data separator as ;

 India SAP CoE, Slide 129 Hints while loading flat file Be sure the delimiter in the InfoPackage is given as ‘ ; ’Be sure the delimiter in the InfoPackage is given as ‘ ; ’ You might get an error on alpha conversion. Take in mind the option in transfer rule to convert the data from external format into internal.You might get an error on alpha conversion. Take in mind the option in transfer rule to convert the data from external format into internal.

 India SAP CoE, Slide 130 Hints while loading flat file Calendar day should be of the format yyyymmdd (SAP requires this format).Calendar day should be of the format yyyymmdd (SAP requires this format). Sometimes you might get an error like the system cannot open the file. Try keeping the file in My Documents or in C drive.Sometimes you might get an error like the system cannot open the file. Try keeping the file in My Documents or in C drive. View the preview option in the InfoPackage before loading the fileView the preview option in the InfoPackage before loading the file Set the number of header rows to be ignored to 1. This ignores the heading row in the flat file Set the number of header rows to be ignored to 1. This ignores the heading row in the flat file

 India SAP CoE, Slide 131 DB Connect, UD Connect and SOAP Based BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 132 Steps involved for DB Connect 1. Create a DB Connect Source System 3. Create InfoSource and assign Source System 4. Maintain communication structure and transfer rules 5. Maintain InfoCube and Update Rules 6. Create InfoPackage 7. Initialize Delta Update 2. Create a DataSource

 India SAP CoE, Slide 133 Steps involved for DB Connect Create a DB Connect Source SystemCreate a DB Connect Source System Create a DataSourceCreate a DataSource Create InfoSource and assign the Source System created to it.Create InfoSource and assign the Source System created to it. Create InfoPackage and load the data.Create InfoPackage and load the data.

 India SAP CoE, Slide 134 Step 1 - Create a DB Connect source system DB Connection : Give the name of the connection. DBMS : Enter the database type to which you want to connect in this field. (i.e. Oracle, DB2 etc) Enter the username and password. Conn. Info : This field contains the technical information that is analyzed in native SQL in the CONNECT statement in order to open the database connection. e.g. In Oracle enter the name of the database you want to connect

 India SAP CoE, Slide 135 Taking over Metadata via DB Connect You use the context menu for a database Source System  Select Database Tables to generate a DataSource for database source systems. First, you choose a selection of tables for a database source system and create a connection to the database source system. Next, you select the table fields for a specific table of the database Source System, and specify whether you want these table fields to be available for selection in the BW system. Step 2 - Generate DataSource

 India SAP CoE, Slide 136 Based on the generated DataSources for external DB views/tables InfoSources in BW can be established as usual. Dataflow is now possible! Step 3 - Assign the DataSource to the InfoSource

 India SAP CoE, Slide 137 DataSource Enhancement BI Content Extractors LO Cockpit FI-SL Extractors CO-PA Extractors Generic Extractors Flat File Extraction DB Connect, UD Connect and SOAP based Datasource Enhancement

 India SAP CoE, Slide 138 Datasource Enhancement via Customer Exit The different customers exits for enhancements of transaction data, master data attributes, texts and hierarchy DataSources areThe different customers exits for enhancements of transaction data, master data attributes, texts and hierarchy DataSources are –EXIT_SAPLRSAP_001 : Transaction data DataSource –EXIT_SAPLRSAP_002 : Master data DataSource –EXIT_SAPLRSAP_003 : Text DataSource –EXIT_SAPLRSAP_004 : Hierarchy DataSource

 India SAP CoE, Slide 139 Datasource enhancement Steps involved 1. Check whether required field is available in extract structure 3. If not available proceed with enhancement 4. Enhance extract structure with append structure 5.Unhide the custom field 6. Write appropriate customer exit for enhancement 2. If available include the same in extract structure

 India SAP CoE, Slide 140 Step 1 - Check for availability in Extract Structure Applicable for DataSources where required field is available but not included by defaultApplicable for DataSources where required field is available but not included by default Pre-requisite is empty delta queue and set-up tablesPre-requisite is empty delta queue and set-up tables

 India SAP CoE, Slide 141 Step 2 - Enhance Extract Structure Find the name of the extract structure for DataSourceFind the name of the extract structure for DataSource Namespace for custom field is ‘YY or ZZ’Namespace for custom field is ‘YY or ZZ’ Activate append structureActivate append structure For currency/quantity fields, the reference field should also be present in the extract structureFor currency/quantity fields, the reference field should also be present in the extract structure

 India SAP CoE, Slide 142 Step 3 - Unhide the added field Note that ‘Field Only’ is checkedNote that ‘Field Only’ is checked Incase field is required for selection, check ‘Selection’Incase field is required for selection, check ‘Selection’

 India SAP CoE, Slide 143 Step 4 - Write the logic Go to CMODGo to CMOD Enter Project name and select componentEnter Project name and select component Select appropriate Customer ExitSelect appropriate Customer Exit

 India SAP CoE, Slide 144 Write the Logic Write the ABAP code for populating the custom fieldWrite the ABAP code for populating the custom field Test Enhancement in RSA3Test Enhancement in RSA3 Replicate DataSource in BWReplicate DataSource in BW

 India SAP CoE, Slide 146 Flat File Extraction Create the InfoObjects document number, order no, name of the material, cost, currency, order creation date in the BW system.Create the InfoObjects document number, order no, name of the material, cost, currency, order creation date in the BW system. Create a flat file containing data for above InfoObjectsCreate a flat file containing data for above InfoObjects Save the file in the CSV format.Save the file in the CSV format. Now create a flat file source system and InfoSource and assign the Source System to the InfoSource.Now create a flat file source system and InfoSource and assign the Source System to the InfoSource. Create an ODS with document number as key field and include the above InfoObjects.Create an ODS with document number as key field and include the above InfoObjects. Create the update rules from the InfoSource you have created.Create the update rules from the InfoSource you have created. Create an InfoPackage and load the file.Create an InfoPackage and load the file.

 India SAP CoE, Slide 147 BI Content Extraction In the Source system go to transaction RSA5, activate the DataSource 2LIS_11_VAITM.In the Source system go to transaction RSA5, activate the DataSource 2LIS_11_VAITM. Go to BI Content select the grouping as data flow before and install the InfoSource 2LIS_11_VAITM.Go to BI Content select the grouping as data flow before and install the InfoSource 2LIS_11_VAITM. Now replicate the DataSource in BW and assign it to the InfoSource and maintain transfer rulesNow replicate the DataSource in BW and assign it to the InfoSource and maintain transfer rules Create an ODS and update rulesCreate an ODS and update rules Create the InfoPackage and load the ODSCreate the InfoPackage and load the ODS

 India SAP CoE, Slide 148 LO Cockpit Go to transaction SBIW. Check out the various options to delete and fill the set up tables and simulate the batch jobsGo to transaction SBIW. Check out the various options to delete and fill the set up tables and simulate the batch jobs Go to transaction LBWE, check the option to maintain the datasources and to activate them as well as to simulate the V3 updateGo to transaction LBWE, check the option to maintain the datasources and to activate them as well as to simulate the V3 update

 India SAP CoE, Slide 149 Transactions RSA1 Administrator Workbench RRMX Business Explorer ( BEX ) RSO2 Maintain Generic Datasource RSA3 Extractor Checker RSA7 Delta Queue Status RSA6 Maintain Datasources RSQ02 Maintain InfoSets RSBOH1 Open Hub Destination

 India SAP CoE, Slide 150 Transactions RSMOMonitoring SBIW BW Config. OLI*BW Statistical Setup SM50 Process Overview RSA5Install Business Content RSA6Post Processing of DataSources LBWELogistics Cockpit LBWGDeleting Setup tables LBWFBW Log SM37 Job Overview KEB0Create the COPA Data Source

 India SAP CoE, Slide 151 Course Name 1 PrepareMe 2 TellMe 3 ShowMe 4 LetMe 5 HelpMe

 India SAP CoE, Slide 152 Tips & Tricks Before starting an SAP BW project, analyze the reporting requirements against the standard DataSources available. Zero-in on the standard DataSources satisfying the requirement. If data that cannot be supplied by the standard sources, we can: - Create a generic DataSources. - Enhance the standard DataSource.

 India SAP CoE, Slide 153 Some Useful sites www.sdn.sap.com www.help.sap.com www.service.sap.com

 India SAP CoE, Slide 1 BI1003 – Data Extraction – v2.0.

Similar presentations

Presentation on theme: " India SAP CoE, Slide 1 BI1003 – Data Extraction – v2.0."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

 India SAP CoE, Slide 1 BI1003 – Data Extraction – v2.0.

Similar presentations

Presentation on theme: " India SAP CoE, Slide 1 BI1003 – Data Extraction – v2.0."— Presentation transcript:

Similar presentations

About project

Feedback