Biztalk EDI 101

There is two example on the net doing unzip pipeline component. I did the zip part and post it here if anybody wants it.
First, go to download the pipeline component wizrad:
http://www.gotdotnet.com/Workspaces/Workspace.aspx?id=1d4f7d6b-7d27-4f05-a8ee-48cfcd5abf4a
Second,
Go to
http://www.gotdotnet.com/workspaces/workspace.aspx?id=ab5544dc-6d5c-4ffa-8ab9-e3dd4954c0cf

Blogspot does not have upload zip file capability. I will put code on this workspace.

Pay attention to the unzip part, it is done as a DisassemblingParser component (so that the GetNext method is available to get next entries in the zip file. A zip file can have many xml files inside itself, remember?
The zip part is a decode component.

For some rare reason, I have this bug happened when compiling biztalk project.
The port type,port and messages are there but the compiler complained that they are missing. The compilation of the project will fail. Looking at the orchestration GUI, there is not RED flag. It is very frustrating because the project was compiled in the last work day and not the current day when you just open it up.

I fixed this by breaking the link between the port and the receive/send shape, then compile.
This will fail and the failure is intentional. Next, re-link the port and the receive/send shape, then compile. Compilation is now successful.

http://www.scottberkun.com/essays/essay40.htm

By default, the chunk size and threshold size are too small for large xml files.
This makes biztalk too "chatty".
Here is the config that help me a lot:

http://www.canonical-ltd.com/publicdoc/Convoys_problems.pdf

http://geekswithblogs.net/abhijeet/archive/2005/05/29/40886.aspx
Cool!

http://news.moneycentral.msn.com/ticker/article.asp?Symbol=US:MSFT&Feed=PR&Date=20050802&ID=5011506
Technology Purchase From iWay Software Enhances BizTalk Server Application Connectivity.

REDMOND, Wash., Aug. 2 /PRNewswire-FirstCall/ -- Microsoft Corp. MSFT today announced the acquisition of eight .NET-based application adapters from iWay Software to be made available by Microsoft in early 2006. This acquisition reflects Microsoft's commitment to BizTalk(R) Server by providing customers with a single architecture, customer support and licensing process. In addition to the adapters in Microsoft's newly expanded portfolio, iWay and other industry partners will continue to deliver additional adapters that address the needs of enterprise customers.

(Logo: http://www.newscom.com/cgi-bin/prnh/20050802/SFTU050 )

"We're committed to making Microsoft(R) BizTalk Server the most complete offering for our customers. Part of that commitment is providing adapters for the most commonly used line-of-business applications and technologies," said Ted Kummert, corporate vice president of the Business Process and Integration Division at Microsoft. "Our existing adapter partners will continue to be a critical part of our strategy to provide total connectivity in complex IT environments."

The adapters Microsoft purchased will help BizTalk Server customers connect more easily to third-party applications from Amdocs, JD Edwards, Oracle Corp., PeopleSoft, Siebel Systems Inc. and TIBCO Software Inc. iWay customers that have purchased licenses for the .NET-based adapters will receive a license for the corresponding Microsoft adapter with the purchase of Software Assurance.

Not as simple as the base EDI 997, which is as easy as turning on a configuration on the GUI.
1. Covast EDI 997 requires the 997 schema to be imported to CovastDB database through the use of Covast Studio.

2. There are "map" text file schemas that go together with the 997 schema. I speculate these are for internal temp text files that bundle many 997 information in an interchange. Think of it like a batch 997 because 997 can be batched at document-level and interchange-level(a batch of documents). I have seen these map file in there run-time dump form. Map files must also be imported using Covast Studio.
-->1. and 2. form a 997 deployment package that can be download off Covast website.

3. There are many version of the deployment package.
For example: 997 4010; 997 4050.
4010, 4050, 4060 versions of the 997s correspond to the versions of the envelope of the inbound edi documents. This point is important because an inbound edi of version 4050 can be enclosed inside an envelope of version 4010. In fact, for example, 850-4050 and 820-4060 can be both enclosed in an envelope of version 4010! There will be a blog about how to configure Covast EDI receive port to allow this.

From MSDN
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/operations/htm/ebiz_prog_adapt_edi_izkk.asp

http://www.theregister.co.uk/2005/06/28/sun_seebeyond/
Sun Microsystems is buying its way into the Service Oriented Architecture (SOA) club dominated by IBM, with the $387m purchase of SeeBeyond Technologies.

As part of the deal, announced early Tuesday, SeeBeyond's 60 sales staff will be used to lead Sun's own sales efforts pitching a range of Sun-branded SOA tools and adaptors to customers. SeeBeyond has a total of 814 employees - most of which are expected to join Sun.

Sun will integrate SeeBeyond's Integrated Composite Application Network (ICAN) suite into the Java Enterprise System (JES) while, Sun said, also continuing to make ICAN available as a separate product. ICAN, which already supports a number of Sun products, will stretch to cover all of Sun's enterprise software products during the next six months.

ICAN is SeeBeyond's integration platform, providing drag-and-drop tools for application, data and business integration, and is capable of working in Java, XML and SQL. ICAN allows customers to build composite applications, an architecture that uses features and data found in other applications and assembled as a service.

SeeBeyond has 2,000 enterprise customers spanning manufacturing and health, with possibly its highest profile deal being the $28m e-record system, integrating data in legacy systems for the UK's National Health Service (NHS).

Jim Demetriades, SeeBeyond founder and chief executive, told The Register: "We are going to lead [Sun's] sales team in the sale of SOAs, and train and teach Sun's sales people."

Scott McNealy, Sun chairman and CEO said the idea mirrored Sun's $4.1bn StorageTek acquisition. "That increased our data management expertise by an order of magnitude. They [SeeBeyond] are the lead dog in this space, without a doubt."

Purchase of an integration company by Sun is long overdue and comes at the point when the industry is turning integration into a SOA story - where applications and data are delivered as services. Until now, Sun filled the integration gap in its portfolio through a technology and marketing partnership with SeeBeyond, to port components of ICAN to JES.

IBM is the largest vendor in this space, according to analysts, having been locked in competition, primarily, with BEA Systems for the top slot. SeeBeyond has been competing against IBM, BEA and smaller vendors like webMethods.

Lacking its own integration product, Sun has led the Java Business Integration (JBI) specification through the Java Community Process (JCP). JBI provides a way for different integration architectures to talk to each other through Java.

Sun announced backing from 19 companies for JBI on Monday, in addition to releasing its Java Enterprise Service Bus (ESB), a reference implementation of JBI, to the community. SeeBeyond will become Sun's JBI implementation, Sun said.

Acquisition of SeeBeyond signals a growing presence by Sun in vertical markets and small and medium businesses (SMBs). Sun will work with Systems Integrators and consultants like Accenture and EDS to build reference architectures to integrate applications and systems in specific markets.

IBM has been successful in taking knowledge from its services organization in specific sectors and wrapping that with WebSphere middleware integration tools to help customers integrate legacy systems and build SOAs.

Demetriades said 34 per cent of SeeBeyond's revenue comes from the health sector, of which a large proportion of customers can be considered SMBs with between $5m and $25m in annual revenue. Around 60 per cent of SeeBeyond's revenue comes from manufacturing, finance and government. ®

Covast Engine Input File Locking (EIF)
Schema validation failed: Repository was locked on host [hostname] by process [xsd2edi], processid [nnn]

Sympton: Build output:
XSD2EDI failed to convert XSD: Can't lock repository because it is already locked
You can google this but let me just quote it from one existing blog:
http://weblogs.ilg.com/brumfieldb/category/54.aspx
The above blogger has some good HIPPA blog too.
---Quote
We've been using the Covast EDI Accelerator for BizTalk 2004 recently as part of a solution for a client. One issue we've run into and have just discovered the solution for is that the Engine Input File (EIF) file may get locked if your process is killed while compiling the EIF file.

There may be legitimate reasons for these to be locked (such as another user actually changing the EIF file), but if you are sure that no one else should be editing the file, you can remove the locking flag manually.

To do so, go to the Covast database. Find the parame table and set the value in repolock column to NULL (Ctrl-0 in Enterprise Manager). I believe there should only be one row in this table, so that will be the row you edit.
--End Quote--

Splitting XML and retain parent and grand parent info.

Based on some splitting XML using Xpath blog on the Net, I have a new variation of it done.
This blog only split a parent, child structure.

I adapted it to have lower bound and upperbound for the xpath loop.
This way, I can split this xml.
GrandParent1
Parent1
Child1
Child2

Into 2 xml files:
GrandParent1
Parent1
Child1

GrandParent1
Parent1
Child2

This just recently helped me to batch some Covast EDI (will be explained in another post) with a very strict requirement.

I will put the source code on soon.

I found this some where around the net:
Very nice explanation!

Prime factorization happens to be the foundation for secure data communications. It’s relatively easy to multiply two prime numbers together (7,817 and 7,333, for example), but no one has found an easy way to do the calculation in reverse — that is, figure out which two prime numbers can be multiplied together to equal 57,322,061.

This is what makes public-key cryptography possible. Other people can send you messages that are coded using the product of two primes, but that secret message can be deciphered only by someone who knows the two prime factors.

Your computer automatically handles all this coding and decoding in a secure electronic transaction. That’s what protects your credit card information from electronic eavesdroppers when you buy something over the Internet. But suppose the eavesdroppers had quantum computers: With all that computing power, they could figure out the prime factors of even incredibly large numbers — and crack the code.

Thus, the development of quantum computers would require a complete change in the methods used to protect information transmitted over the Internet and other “secure” communications links.

Solution 3:
Compile the EDI schema in a separate dll (and deploy it alone).
The EDI 820 for example will have 20 megs dll.
But after that, your biztalk EDI project just reference this 20 megs dll and things will be fine.

Biztalk Covast EDI problem: Internal Compiler Error
...likely culprit is 'CODEGEN'.

Internal Compiler Error (0xc0000005 at address 5362514E):
likely culprit is 'CODEGEN'.

This has been confirmed by Covast to be an VS.Net internal
error. The cause is the size of the xsd is too large(10 megs and above is
considered large).

Solution 1:(not always works, not always reducing the size
enough)

Inside Visual Studio.Net, right click the biztalk project,
click on add generated items->generate schema->generate Covast EDI
schema.

Uncheck the "include description". By unchecking
this, the description will not be included in the tag name of the covast edi
xsd. Therefore; the of the tag size is reduced=> the size of the whole xsd
is reduced.

 

Solution 2: (should be always works).

Rather messing with the codeset inside Covast EDI
database(CovastDB), we can go to the large xsd, try to manually reduce its size
by deleting repeating and long enum collection.

DELETE THOSE ENUM, only LEAVE THE ENUM YOU NEED.

Look for these nodes:

REDUCE THESE FIELDS' enum:

 

Reference_Identification_Qualifier

Identification_Code_Qualifier

Unit_or_Basis_for_Measurement_Code

Entity_Identifier_Code

Date_Time_Qualifier

Location_Qualifier

Product_Service_ID_Qualifier

Agency_Qualifier_Code

Date_Time_Period_Format_Qualifier

Amount_Qualifier_Code

Contact_Function_Code

Try later:

Service_Level_Code

Surface_Layer_Position_Code

 

For example:

<xs:simpleType
name="Reference_Identification_Qualifier">

<xs:restriction base="CovastTypeA">

<xs:minLength value="2"/>

<xs:maxLength value="3"/>

<xs:enumeration value="01"/>

<xs:enumeration value="02"/>

<xs:enumeration value="03"/>

<xs:enumeration value="04"/>

<xs:enumeration value="05"/>

<xs:enumeration value="06"/>

<xs:enumeration value="07"/>

<xs:enumeration value="08"/>

style='mso-spacerun:yes'> <xs:enumeration value="09"/>

<xs:enumeration value="0A"/>

<xs:enumeration value="0B"/>

<xs:enumeration value="0D"/>

<xs:enumeration value="0E"/>

<xs:enumeration value="0F"/>

<xs:enumeration value="0G"/>

<xs:enumeration value="0H"/>

<xs:enumeration value="0I"/>

............................

Replace it with only the enum you need:

<xs:simpleType
name="Reference_Identification_Qualifier">

<xs:restriction
base="CovastTypeA">

<xs:minLength
value="2" />

<xs:maxLength
value="3" />

<xs:enumeration value="AM" />

<xs:enumeration value="IK" />

<xs:enumeration value="IV" />

<xs:enumeration value="PO" />

<xs:enumeration value="ST" />style='mso-spacerun:yes'>

</xs:restriction>

</xs:simpleType>

 

Repeat the process over and over again with other type of
nodes.

Note: one type of nodes can appear many times in the same
xsd.

 

# posted by hung @ 1:06 PM 1 comments

This is my first post:)