Free Response to Motion - District Court of Colorado - Colorado

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 1 of 12

IN THE UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLORADO Civil Action No. 04-cv-1258-LTB-BNB STUDENT MARKETING GROUP, INC., Plaintiff, v. COLLEGE PARTNERSHIP, INC., f/k/a COLLEGE BOUND STUDENT ALLIANCE, INC., Defendant. _____________________________________________________________________________ DEFENDANT'S RESPONSE IN OPPOSITION TO PLAINTIFF'S MOTION IN LIMINE TO EXCLUDE EXPERT TESTIMONY ______________________________________________________________________________ Defendant College Partnership, Inc. ("College Partnership"), by and through its counsel, Rosemary Orsini and Brian K. Matise of Burg Simpson Eldredge Hersh & Jardine, P.C., hereby submits its response to Plaintiff Student Marketing Group, Inc.'s "Motion in Limine to Exclude the Expert Testimony of Mr. Nathan Allen and Mr. Thomas Hiller." Plaintiff's motion should be denied because Mr. Hiller and Mr. Allen utilized well-accepted computer programs and techniques in conducting their analysis. Mr. Hiller also conducted internal reliability checks (such as sampling the original data files) to insure accuracy. Plaintiff's hyper-technical objections go to the weight of the testimony of these witnesses, not its admissibility. Accordingly, the Court should deny the motion. In the alternative, the Court should conduct a hearing pursuant to Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993) and Fed. R. Evid. 104 and allow Mr. Hiller and Mr. Allen to explain their methodologies to the Court. I. INTRODUCTION Plaintiff's Motion in Limine seeks to exclude the testimony of Mr. Nathan Allen and Mr.

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 2 of 12

Thomas Hiller. Mr. Allen is a computer programmer specializing in direct mail marketing data processing. Mr. Allen has three years of experience as a programmer, including experience with Merrell Remington, a direct mail marketing firm that is responsible for large advertising mailings. See Resume of Nathan Allen (Exhibit 8 to Plaintiff's Motion in Limine at p. 8). Mr. Allen's duties during that three year period of time includes running standard "CASS certification" programs on mailing lists to insure that the addresses in the lists agree with the United States Postal Service database of valid addresses, and therefore have a high probability of deliverability. See Deposition of Nathan Allen (Exhibit 5 attached to Plaintiff's Motion in Limine at pp. 16-17). This technique prevents large amounts of wasted mailings that would result if advertising mailings were sent to nonexistent addresses or addresses that were so incomplete that they could not be delivered. Mr. Allen also performs "merge/purge" processes that remove duplicate names that may exist in a mailing when multiple address lists are combined. These so-called "de-duping" processes avoid sending two, three, or more duplicate mailings to the same person. Mr. Allen is a hybrid fact/expert witness who will not be offering an opinion as such. Instead, he will be called to testify that he ran certain standard programs on College Partnership's actual mail files in 2004 to determine the number of student records for each vendor that either failed the "CASS certification" process (indicating an address with errors that has a high probability of being undeliverable) or were duplicates. See Deposition of Nathan Allen (Exhibit 5 at p. 13 lines 19-22). Accordingly, he will testify only as to his actions in running these programs that he is well familiar with, and explaining the output of these programs. Out of an abundance of caution, College Partnership designated Mr. Allen as an expert witness and provided a complete expert report from Mr. Allen in accordance with Fed. R. Civ. P. 26(a)(2). -2-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 3 of 12

Mr. Allen's results were then provided to Mr. Tom Hiller, an expert in the field of direct mail advertising. Mr. Hiller has almost 40 years of experience in direct mail marketing, data acquisition, mail list management, and information management. See Resume of Tom Hiller, (Exhibit 6 attached to Plaintiff's Motion in Limine, at p. 5-8). Mr. Hiller will offer the opinion that the lists Student Marketing Group provided to College Partnership were not "98% accurate" as advertised, had a high rate of addresses that were not "deliverable" according to industry standards, and had a high level of duplicate records for which College Partnership was charged. See Exhibit 6 at pp. 1-4, Exhibit 7 (Deposition of Tom Hiller, attached to Plaintiff's Motion in Limine) at p. 18 lin1 1 - p. 19 line 13. Hiller will also testify that the list quality of the records SMG supplied to College Partnership in 2004 was among the worst quality of any list provider, as opposed to SMG's claim that its lists were among the best in the industry. See Deposition of Tom Hiller (attached as Exhibit 7 to Plaintiff's Motion in Limine, at p. 17 line 23 - p. 18 line 16).

II. FACTUAL RESPONSES TO PLAINTIFF'S CHALLENGES Plaintiff takes a number of "pot shots" at Mr. Hiller's and Mr. Allen's analysis without any showing whatsoever that these challenges affect the reliability of the data or the results. Plaintiff presents no affirmative evidence from any expert that these challenges make any difference whatsoever in the reliability analysis. Accordingly, these challenges are more appropriate as cross examination questions challenging the weight of the expert opinion rather than its admissibility. Plaintiff challenges the opinions of Mr. Hiller and Mr. Allen on a number of grounds, including the following: -3-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 4 of 12

·

Plaintiff challenges the data analyzed because Mr. Allen and Mr. Hiller utilized College Partnership's actual mail files instead of SMG's files. College Partnership no longer has access to SMG's files, and SMG has not come forward in discovery with any of its files in the 2003-2004 period; accordingly, if SMG is not hiding discoverable evidence, the Court must conclude this data does not exist. Nonetheless, College Partnership does have its mail files, which contained the actual names and addresses obtained from SMG and other vendors, with a "reservation code" that identified which vendor supplied which name and address. SMG provides no reason why these mail files do not accurately reflect the names and addresses that SMG provided and are contained in these mail files.

·

Plaintiff challenges the data because another former vendor of College Partnership, known by the trade name "PrimeNet," handled the data between the time SMG provided the lists and College Partnership performed the mailings. In fact, all that PrimeNet did was to add a "reservation number" to each record to identify which list provider was the source of the record, merge all of the lists into a single mailing list, perform a "merge/purge de-duping" process to eliminate duplicate records, and perform a "CASS certification" on the records. See Affidavit of Tom Hiller (Exhibit A) at paragraph 7. None of these processes would change the names or the addresses in any way, and Mr. Hiller has no indication from SMG or any other source to indicate that reliability was affected by these processes. Exhibit A at paragraph 8. In fact, the "de-duping" and "CASS certification" process would actually make SMG's data appear better than it was, because if performed, these procedures would "cull out" bad SMG records. In fact, College Partnership in a separate lawsuit is engaged in litigation against PrimeNet -4-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 5 of 12

because College Partnership learned, at the same time it learned of the problems in the SMG lists, that PrimeNet was not performing the CASS certification and "merge/purge de-duping" so as to eliminate these records; accordingly, College Partnership wasted hundreds of thousands, or even millions, of dollars on mailings to undeliverable or duplicate addresses. · Plaintiff challenges Mr. Allen's qualifications to perform the programming; however, Mr. Allen has been programming in this area for three years. The program used, Satori, is the program that Merrell-Remington uses to certify all their mailings. Mr. Allen has used that particular program regularly for a year. See Deposition of Nathan Allen (Exhibit 5 to Plaintiff's Motion in Limine, at p. 13 line 19 to p. 17 line 1. · Plaintiff contends that Mr. Hiller's opinions are unreliable because Mr. Hiller offered opinions before Mr. Allen completed his analysis. This mischaracterizes what actually occurred. Mr. Allen completed his processing of data and supplied the results to Mr. Hiller. Mr. Hiller performed some reliability checks on the data, and identified a few errors in some of the records. As a result of this check, Mr. Allen revised his processing to account for this and re-processed the data. See Affidavit of Tom Hiller (attached hereto as Exhibit A, paragraph 6). The impact of this error that Mr. Hiller caught was completely negligible - it affected "a hundred records out of eight million." Deposition of Tom Hiller, Exhibit 7 to Plaintiff's Motion in Limine, at p. 60 lines 11-19. Instead of casting doubt on the analysis, the fact that Mr. Hiller utilized reliability checks that caught an error that affected perhaps 100 records out of 8 million demonstrates Mr. Hiller's care and scrutiny. -5-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 6 of 12

·

Plaintiff complains that Mr. Hiller analyzed over 8 million records, including over 1.3 million SMG records. Mr. Hiller analyzed College Partnership's actual mailings for a 23week period in 2004 because these mail files were available. See Affidavit of Tom Hiller (Exhibit A) at paragraphs 4, 7. Mr. Hiller is an expert in the field of information management and analysis (Exhibit A at paragraph 2) and his experience includes sampling large demographic databases. He has offered his opinion based on his experience and opinion that the 8 million record sample is sufficient, and SMG provides no reason why 8 million records is an insufficient sample.

·

Plaintiff complains that Mr. Allen performed his CASS certification based on the postal service database of addresses that existed as of November/December 2004 even though the records were mailed out in January - June 2004. This would generate an error only if an address existed in January to June of 2004, but then no longer existed in November/December 2004 - for example, if a building were demolished. Plaintiff is correct that both Mr. Hiller and Mr. Allen could not estimate a rate of error for this unusual hypothetical - however, it would certainly affect all lists the same and is unlikely to have any significant impact, given how rare it would be for an existing address to simply disappear from the post office's database.

·

Plaintiff contends that Mr. Allen's duplicate analysis would underestimate the number of duplicates by failing to consider the possibility that a record from one list could have two different "reservation code" letters. Plaintiff acknowledges this would affect duplicates of only two vendors, American Student Lists ("ASL"), and SMG, which were the only vendors assigned two codes. Moreover, this underestimate would mean that SMG's data -6-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 7 of 12

would actually contain more duplicates and therefore the list would be of even lower quality. III. AFFIRMATIVE FACTS IN SUPPORT OF THE RELIABILITY OF THE ANALYSIS The data analysis performed by Mr. Hiller, with programming assistance by Mr. Allen, is sufficiently reliable to meet the standards of Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 589-90 (1993). Mr. Allen is an experienced computer programmer who has utilized CASS certification programs in his employment with Merrell-Remington, a direct mail marketing service, that uses these programs routinely to certify all their mailings. Exhibit 5 to Plaintiff's Motion in Limine at p. 16 line 15 - p. 17 line 1. CASS certification programs are routinely used in the direct mail industry to determine the probability that an address is valid and deliverable. See Exhibit A at ¶¶ 5. The analysis was performed on mail lists that represented College Partnership's actual mailings for 23 weeks in 2004 during which SMG provided records. (Exhibit A at ¶ 4). These mail lists were compiled by College Partnership's mailhouse, PrimeNet, and consisted of the names and addresses supplied by all of the list vendors merged into a single file, with a "reservation number" appended to it that allowed the records to be identified as coming from SMG or any other particular vendor. See Exhibit A at ¶ 7; Deposition of David Russel (Exhibit B) at page 50 line 24 to p. 51 line 25. PrimeNet performed a "merge-purge" de-duping on the files to eliminate duplicate names and addresses, performed CASS address certification, and appended the reservation number, but the process would not change the names and addresses. Exhibit B (Deposition of David Russell) at p. 74 line 25 - p. 79 line 19. Over 8 million records were analyzed by Mr. Allen and Mr. Hiller. Exhibit A at ¶ 4. -7-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 8 of 12

The CASS certification program yields an error code; the greater the number, the more severe the error in the address and the less likely it can be delivered. CASS error codes greater than 99 (i.e., 100 or more) were considered to be "undeliverable," as was the general rule of the CASS certification utilized by Merrell Remington and the direct mail industry. See Affidavit of Tom Hiller (Exhibit A) at ¶ 4, 6; Deposition of Nathan Allen (Exhibit 7) at p. 14 line 4 to p.16 line 19. Mr. Hiller performed a number of quality control checks on the data, and in the process of doing so, he detected an error in the programming that affected perhaps 100 records out of more than 8 million. Nonetheless, Mr. Allen repeated his calculations to correct this minor error. Exhibit A at ¶ 6; Exhibit 7 at p. 60 lines 11-19. Mr. Hiller had numerous conversations with David Russell of College Partnership, the person at College Partnership who obtained the lists from SMG and other vendors and was most familiar with the processing performed on them, in order to understand the nature of the data records he was utilizing. Exhibit B at p. 78 lines 1 20; Exhibit A at ¶ 7. III. ANALYSIS

As Plaintiff correctly observes, Fed. R. Evid. 702 requires that an expert witness may offer opinion testimony if three elements are satisfied: (1) the testimony must be based on sufficient facts or data; (2) the testimony must be the product of reliable principles and methods; and (3) the witness applied the principles and methods reliably to the facts of this case.

-8-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 9 of 12

The Court has the responsibility to make the determination as to whether these elements have been satisfied and the expert testimony should be admitted. See Fed. R. Evid. 104(a). The Court has considerable discretion in performing this "gatekeeper" function, see, Goebel v. Denver & Rio Grande W. R.R., 346 F.3d 987, 990 (10th Cir. 2003), but the Court must make specific factual findings on the record which are sufficient for an appellate court to review the findings. Dodge v. Cotter Corp., 328 F.3d 1212, 1223 (10th Cir. 2003). In order to establish that an expert's testimony is reliable, the party offering the evidence need not prove that the expert is undisputably correct. Bitler v. A.O. Smith Corp., 400 F.3d 1227, 1233 (10th Cir. 2004). "When examining an expert's opinion ... the inquiry should not be aimed at `the exhaustive search for cosmic understanding, but for the particularized resolution of a legal dispute.' Thus it is the specific relation between an expert's method, the proffered conclusions, and the particular factual circumstances of the dispute, and not asymptotic perfection, that renders testimony both reliable and relevant." Id.at 1233-34 (quoting Daubert, 509 U.S. at 597). College Partnership has established the requisite reliability of Mr. Allen and Mr. Hiller's methodology. As the Tenth Circuit Court of Appeals noted in Bitler, the key issue for the Court is whether the methods of Mr. Allen and Mr. Hiller, when applied to this particular legal dispute,are sufficiently reliable to make it "more likely than not" that SMG's data is of lower quality than its competitors and not "98% accurate." See id. at 1234-35. Although "greater certainty" may be possible through further tests and analysis on 24 million records instead of 8 million records, "such a high degree of certainty is not required" in conducting a Daubert analysis. Id. at 1236. Daubert and Kumho Tire Co. v. Carmichael, 526 U.S. 137, 150, 152-53 (1999), have -9-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 10 of 12

established several factors that the Court may consider in conducting its reliability analysis. These include: (1) whether a theory has been tested; (2) whether the theory has been subjected to peer review; (3) whether there are known or potential rates of error; and (4) whether the theory or technique is generally accepted. Daubert, 509 U.S. at 593-94. The methodology used by Mr. Allen and Mr. Hiller satisfies these tests. First, the CASS certification software used in this case is industry-standard software utilized by the direct mail marketing industry, based on the United States Postal Service's address database, to determine the deliverability of mail. This satisfies the testing of this technique (by multiple vendors for billions of pieces of mail), as well as general acceptance in the industry. Second, the methodology used by Mr. Allen and Mr. Hiller was tested and verified through an extensive series of random checks, through which an error affecting only about 100 records out of over 8 million was detected. If Mr. Hiller's error analysis was so thorough that it could detect errors on this scale, there is a high probability that more significant errors would have been detected. Mr. Allen's duplicate analysis also is well established. He simply compared addresses that were identical, then determined using the U.S. Bureau of the Census SOUNDEX procedure whether the names were the same. See Exhibit 8 at p.p. 5-7. Once again, the technique is generally accepted and has been subjected to extensive use by the U.S. Census Bureau and others. The data analyzed is also sufficiently reliable in quantity and quality. Mr. Allen and Mr. Hiller analyzed over 8 million student records; Mr. Hiller concluded this was sufficient to base his opinions. The records were from the actual mailing lists used by College Partnership for -10-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 11 of 12

2004. Although Plaintiff argues to the contrary, there is no evidence that the technique used by PrimeNet of simply appending a "reservation code" on the records affected the names or addresses in any manner. Finally, PrimeNet's CASS certification and "merge-purge" de-duping would only improve the performance of SMG's lists by removing bad records; it would not affect it negatively. Mr. Hiller and Mr. Allen also correctly applied the methodology to the data. As indicated above, the methodology was applied to the mailing lists in exactly the same manner as MerrellRemington performed the CASS certification and merge-purge on its client's actual mailings. Finally, Mr. Hiller, as noted above, conducted reliability tests to insure accuracy.

IV. CONCLUSION

For the reasons indicated, the Court should deny Plaintiff's Motion in Limine. In the alternative, the Court should conduct a hearing pursuant to Fed. R. Evid. 106 and Daubert v. Merrell Dow Pharms. Inc., 509 U.S. 579 (1993). Respectfully submitted, BURG SIMPSON ELDREDGE HERSH & JARDINE, P.C.

Brian K. Matise 40 Inverness Drive East Englewood, Colorado 80112 Telephone: (303) 792-5595 Facsimile: (303) 708-0527 ATTORNEYS FOR DEFENDANT COLLEGE -11-

Case 1:04-cv-01258-LTB-BNB

Document 100

Filed 07/14/2005

Page 12 of 12

PARTNERSHIP, INC. CERTIFICATE OF SERVICE I hereby certify that on July 14, 2005 I electronically served this document on the undersigned counsel by the CM/ECF system: Gary Parish, Esq. R. Daniel Scheid, Esq. Sander, Scheid, Ingebretsen, Miller & Parish P.C. 700 17th St., Suite 2200 Denver, CO 80202 Patrick J. McElhinny, Esq. Dianna S. Karg, Esq. Kirkpatrick & Lockhart LLP 535 Smithfield St. Henry W. Oliver Building Pittsburgh, PA 15222 _________________________________ /s/Brian K. Matise

-12-