Free Motion in Limine - District Court of Colorado - Colorado

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 1 of 14

IN THE UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLORADO Civil Action No. 04cv-1258-LTB-BNB STUDENT MARKETING GROUP, INC., Plaintiff, v. COLLEGE PARTNERSHIP, INC., f/k/a COLLEGE BOUND STUDENT ALLIANCE, INC., Defendant. MEMORANDUM IN SUPPORT OF PLAINTIFF STUDENT MARKETING GROUP, INC'S MOTION IN LIMINE TO EXCLUDE THE EXPERT TESTIMONY OF MR. NATHAN ALLEN AND MR. THOMAS HILLER Plaintiff Student Marketing Group, Inc. ("SMG"), by and through its undersigned counsel, files this Memorandum in Support of Plaintiff Student Marketing Group Inc.'s Motion In Limine to Exclude the Expert Testimony of Mr. Nathan Allen and Mr. Thomas Hiller. I. INTRODUCTION

Pursuant to a May 19, 2003 List Rental Renewal Agreement (the "Agreement"), SMG agreed to rent millions of names and addresses from its student database to College Partnership, Inc. ("CPI") for CPI to use in its direct mail marketing efforts. In a transparent attempt to avoid payment of its final three monthly payments due to SMG under the Agreement, CPI has counterclaimed against SMG, complaining about the quality and quantity of the names and addresses provided by SMG. CPI has offered multiple experts in support of its claims,

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 2 of 14

including Mr. Nathan Allen as a purported expert in data processing and Mr. Thomas Hiller as a purported expert in direct mail. Astonishingly, however, neither Mr. Allen nor Mr. Hiller has ever reviewed the actual data provided by SMG. Instead, Mr. Allen reviewed the data only after it was manipulated by CPI's "mailhouse," a company named PrimeNet that took SMG's data, combined it with data from CPI's other list vendors and performed the actual mailings for CPI. CPI has not and cannot establish that the information analyzed by its purported experts was the same as the information provided by SMG because not only does CPI concede that PrimeNet manipulated the SMG data and combined it with data from other list vendors, but CPI also is suing PrimeNet for not performing those services properly. See Ex. 1, CPI's Counterclaim against PrimeNet at ¶¶ 8, 15, 19, 24, 29, 30, 30. Furthermore, the analyses (or lack thereof) undertaken by Messrs. Allen and Hiller are seriously flawed. The expert testimony of Mssrs. Allen and Hiller should be excluded because it is based on (1) unreliable and insufficient data and (2) the applications of an unreliable methodology to the facts of the case. II. A. Background CPI is in the business of renting addresses from companies such as SMG to facilitate its direct mail program to potential customers. SMG is in the business of leasing student names and other data elements to customers such as CPI. Pursuant to the Agreement, SMG agreed to lease CPI up to 9 million high school records. To execute its direct mail program, however, CPI used several list vendors in addition to SMG (as many as eight other vendors), including American Student List ("ASL"), which, in fact, supplied CPI with more FACTS

2

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 3 of 14

names than SMG supplied.1 CPI also retained PrimeNet to act as its "mailhouse." PrimeNet's services included combining the information provided by the various list vendors, applying codes that purportedly allowed CPI to attribute the source of the names and addresses to a particular vendor, eliminating duplication arising from names that appeared on more than one vendor's list, running the names and addresses through a computer program designed to ensure the addresses were recognized by the Post Office and preparing the direct mail packages and send the mail. See generally Ex 1. When CPI refused to pay PrimeNet for its services, PrimeNet sued to collect the debt, and CPI counterclaimed for breach of contract, negligent misrepresentation and concealment ­ exactly as it did in this case. Among other things, CPI alleges that PrimeNet failed to perform the list "cleansing and hygiene properly." See, e.g., Ex. 1 at 15, 39.2 These allegations, made by the same law firm representing CPI in this case, cast serious doubt on the integrity of the database of names and addresses as it existed at the time it was actually used for mailing. B. Nathan Allen Nathan Allen is a newly-minted computer programmer just two years out of college; he is not an expert in direct mail. See Ex. 5, Allen Dep. at 21, 29. CPI retained Mr. Allen as its data processing expert to take several data files and run them through two separate computer programs that were designed to (1) "CASS certify" the files and (2) to check for

ASL and SMG were CPI's largest list vendors. CPI has conceded that, of all of its list vendors, SMG was among the best performers. See Ex. 2, Russell Dep. at 47-48, 100-01; Ex. 3 Grace Dep. at 61; Ex. 4 Anderson Dep. at 31, 107, 110. The elimination of duplicates and addresses not recognized by the Post Office has been described as list "cleansing and hygiene." 3
2

1

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 4 of 14

duplicate names and addresses.3 Mr. Allen's opinions are limited to the fact that he ran the programs and obtained results. Ex. 5, Allen Dep. at 13, 19, 20. Although he has some limited experience running the CASS-certification program, Ex. 5, Allen Dep. at 24, 29, Mr. Allen had never before sorted duplicates from a database using the program that he used in this case; his boss (who has not been designated by CPI) apparently selected the program and instructed Mr. Allen as to its use. Ex. 5, Allen Dep. at 35-37. Mr. Allen did no analysis to determine whether the program to identify duplicate names was appropriate for the purpose, was not aware of anyone else who used it for that purpose and was not aware of its error rate. Ex. 5, Allen Dep. at 111. Likewise, Mr. Allen knew nothing about the source of the data that he analyzed, whether it contained data from multiple vendors, who applied the "identifier codes" (a/k/a "reservation numbers") that he used to categorize the data or whether the data had been manipulated by PrimeNet before it had been provided to him, Ex. 5, Allen Dep. at 30-31, 59. Moreover, there are questions about how Mr. Allen even obtained the data that he did analyze since Mr. Allen testified that Mr. Russell of CPI provided the files to him, Allen Dep. at 30-31, but Mr. Russell testified that he never heard of Mr. Allen and did not give him the data that Mr. Allen claims to have analyzed. Ex. 2, Russell Dep. at 34, 54. C. Thomas Hiller Thomas Hiller is the President of Consultants in Information Management, Inc. CPI retained him to analyze the results provided by Mr. Allen and to comment generally on the accuracy of SMG's marketing materials. See generally Ex. 6, Hiller Reports dated 1/12/05 and 2/11/05; Ex. 7, Hiller Dep. at 170. Mr. Hiller, however, rendered his opinions and testified in his deposition before Mr. Allen finally completed his analyses, although he did review some of Mr. "CASS certification" is a process designed to check the addresses in the file against those recognized by the Post Office. 4
3

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 5 of 14

Allen's preliminary analyses. In fact, he identified multiple errors in those analyses and requested that Mr. Allen rerun his programs to redo his analyses. Ex. 6, Hiller Dep. at 58-60; 2/11/05 Report at 2; Ex. 5, Allen Dep. at 70-73. SMG's repeated attempts to schedule an additional deposition of Mr. Hiller to address Mr. Allen's updated analysis have been unsuccessful. In addition to his review of Mr. Allen's preliminary analysis, Mr. Hiller also reviewed SMG's marketing brochure and opined that "SMG's 98% accuracy and update frequency statements and deliverability guarantee is misleading and not what list compilers and marketers do as a standard practice." Ex. 6, 1/12/05 Report at 2. Mr. Hiller, however, conceded: · · · · · · The SMG guarantee in its entirety is not misleading. Ex. 7, Hiller Dep. at 95-96. He had no information about actual undeliverable mail sent by CPI. Ex. 7, Hiller Dep. at 81. He has no opinion as to the actual deliverability of mail to the addresses on SMG's lists. Ex. 7, Hiller Dep. at 88. He had no information about SMG's actual list hygiene and list updating practices. Ex. 7, Hiller Dep. at 82-85, 88. He does not know what percentage of the SMG names and addresses were not deliverable. Ex. 7, Hiller Dep. at 112-13. The fact that an address generates a CASS-certification error code during Mr. Allen's analysis does not mean that it is undeliverable, and he has no information on the percentage of such addresses that are not deliverable. Ex. 7, Hiller Dep. at 57, 115.

These astonishing admissions clearly call into question the reliability of any opinion proffered by Mr. Hiller regarding SMG's lists, let alone an opinion that SMG's lists are inaccurate and its practices are not what list compilers do as standard practice.4

In this regard, Mr. Hiller's opinion is more than a little bit confusing. He testified that SMG's list quality is worse than other vendors, Ex. 7, Hiller Dep. at 17, but had not looked at the 5

4

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 6 of 14

Having ignored or failed to investigate SMG's actual practices, Mr. Hiller relies exclusively upon generalized statistical data to support his opinion that SMG's "98% accuracy" statement is misleading. Specifically, he cites data from the U.S. Census Bureau which shows that 3.9% of the population moves every quarter, thereby likely undermining the accuracy of SMG's database. Mr. Hiller did no research to determine if that statistic applies to families with high school age children (the relevant group here). Ex. 7, Hiller Dep. at 104-7. Furthermore, Mr. Hiller's analysis does not hold true if the party sending the mail (CPI here) chooses to send the mail so it would be forwarded to those who fill out a change of address form. Ex. 7, Hiller Dep. at 90-94, 199-200. D. The Data Reviewed Although Mr. Allen did not know the source of the data that he analyzed, there is no doubt that the data analyzed was not in the same form as it was when SMG provided it to CPI. Mr. Hiller conceded that the analysis done by Mr. Allen was conducted on the data after it had been "manipulated" by PrimeNet and that this would change the data from what SMG had provided. Ex. 7, Hiller Dep. at 100; Ex. 2, Russell Dep. at 51-52. In fact, when asked whether the information supplied by CPI's list vendors such as SMG would be different then the files Mr. Allen claimed he analyzed, Mr. Anderson responded, "definitely." Ex. 4, Anderson Dep. at 163. Additionally, since PrimeNet combined the data of other vendors with SMG's, it is critical that the assignment of the source code or "reservation number" be done correctly to properly attribute a name and address to the vendor who supplied it. Mr. Hiller had concerns about whether PrimeNet had done this properly. Ex. 7, Hiller Dep. at 25-27. Neither CPI nor its

percentage of duplicates supplied by vendors other than ASL and, on a percentage basis, SMG's duplicates were lower than ASL's duplicates. Id., at 80. Hiller also opines that there is no industry standard of deliverability or duplicates. See Ex. 6, 1/12/05 Report; Ex. 7, Hiller Dep. at 74-75. 6

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 7 of 14

experts have any evidence that the data provided by SMG is the same data analyzed by its experts and that the critical source codes or reservation numbers were applied properly. To the contrary, CPI is now suing PrimeNet for failing to properly "cleanse and hygiene" the database of all vendors' names and addresses. See Ex. 1. E. Flaws in Methodology There are at least four serious flaws in the various methodologies used by CPI's experts. 1. Mr. Allen's CASS-Certification Opinions

To determine the number of CASS-certification errors generated by the addresses that he analyzed, Mr. Allen used a program that is regularly (monthly or every other month) updated by the Post Office to reflect changes in addresses that are recognized, such as new streets, new houses, changes in zip codes, etc. Ex. 5, Allen Dep. at 41-44. Mr. Allen used the program as updated for November/December 2004, even though the data analyzed was generated for mail beginning in January 2004. Id. Messrs. Allen and Hiller conceded that this was inappropriate, that the results would be different had Mr. Allen used the proper version, and that using the inappropriate version generated an unknown error rate. Id.; Ex. 7, Hiller Dep. at 12426. 2. The Analysis of Duplicates

When PrimeNet assigned reservation numbers to names and addresses provided by vendors to allow CPI to track the source of names, it assigned the letters A through E to records provided by ASL and F and G to records provided by SMG. Ex. 7, Hiller Dep. at 24; Ex. 2, Russell Dep. at 27-29, 60. Mr. Allen concedes that his analysis failed to take into account that one vendor was assigned multiple letters. Ex. 5, Allen Dep. at 116-21. In so doing, his analysis fails to determine whether there were duplicate records provided by ASL but assigned different 7

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 8 of 14

letters in the reservation number standard. Because ASL had five different letters assigned to it, this could amount to a significant number of duplicates. Mr. Allen further conceded that he would not capture all of the duplicates provided by each vendor and would, for example, undercount duplicates provided by ASL "by who knows how much." Ex. 5, Allen Dep. at 11621. Furthermore, the letter assigned depended, in part, upon whether, for example, an ASL record was duplicated by an SMG record, in which case it was assigned a letter reflecting that it was a "common" name as opposed to a "unique" name. ASL was given "credit" for all the names it had in common with SMG. Ex. 2, Russell Dep. at 27-29. Mr. Allen's analysis does not take this into account. Id.; Ex. 8, Allen's Report. 3. The Analysis Encompassed Only a Fraction of The Names Provided

Mr. Hiller and Mr. Allen considered only a fraction of the data SMG provided to CPI during a 12 month period covered by the Agreement. SMG supplied CPI with nearly 9 million names under the Agreement. Messrs. Hiller and Allen studied only a five month portion of that period. Additionally, SMG supplied CPI with 3.38 million names during the five month period analyzed by Mr. Hiller and Mr. Allen. The analysis of Mr. Hiller and Mr. Allen, however, attributes only 1.32 million names to SMG. Ex. 7, Hiller Dep. at 69, 74-75. They have offered no opinions that it would be proper to extrapolate from this fraction to the whole. 4. Mr. Hiller's Opinions on the Marketing Materials

Mr. Hiller applied no discernable methodology in rendering his opinion that SMG's marketing materials are misleading. He admittedly did no research regarding the actual records supplied by SMG. His opinions are concededly based upon fragments of the materials as he testified that the guarantee is not misleading when viewed in its entirety.

8

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 9 of 14

III.

ARGUMENT

Mr. Hiller's and Mr. Allen's Opinion Testimony Should Be Excluded Federal Rule of Evidence 702 governs the admissibility of expert testimony. Rule 702 provides that: If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert . . . may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case. FED. R. EVID. 702. In applying Rule 702, the court is required to perform a "gate-keeping" function to determine whether the opinion rendered is "reliable." See Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 589-90 (1993). Since the court's "gate-keeping" function applies to all expert matters that fall under Rule 702, courts should use a "flexible" approach to assess reliability. See Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 141-2 (1999). There are several touchstones that should be considered as the Court performs its "gate-keeping" function here. Among other things, Rule 702 requires that an expert's testimony be based upon "sufficient facts or data." FED. R. EVID. 702. In an analysis of whether the facts relied upon are sufficient, the proposed expert's opinion must be based upon facts that satisfy the reliability requirement of Rule 702. See Mitchell v. GenCorp, Inc., 165 F.3d 778, 781 (10th Cir. 1999). Where the data underlying expert opinions and testimony is unreliable and insufficient, their testimony should be excluded. See Lantec, Inc. v. Novell, Inc., 306 F.3d 1003, 1025-26 (10th Cir. 2002) (holding that the District Court did not err in excluding the testimony of an expert who used unreliable data); The Ninth Ave. Remedial Group v. Allis-Chalmers Corp., 141 F. Supp. 2d 957, 960 (N.D. Ind. 2001) (excluding expert testimony that relied upon "data categorizations, compilations, and conclusions" that were unreliable). 9

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 10 of 14

Likewise, Rule 702 requires that the expert's methodology be reliable. See Mitchell, 165 F.3d at 781. In evaluating the methodology employed by an expert, the court should consider whether the principles and methods have been properly applied to the facts. See id. at 782-83. Under Rule 702, "any step that renders the analysis unreliable . . . renders the expert's testimony inadmissible. This is true whether the step completely changes a reliable methodology or merely misapplies that methodology." See id. at 782 (quoting In re Paoli R.R. Yard PCB Litig., 35 F.3d 717, 745 (3rd Cir. 1994)). Finally, testimony which constitutes "ipse dixit" of the expert should be excluded. See Mitchell, 165 F.3d at 782. An expert cannot establish a fact merely by saying so. See In re Breast Implant Litigation, 11 F. Supp. 2d 1217, 1234 (D. Colo. 1998) (citing Grimes v. Hoffmann-LaRoche, Inc., 907 F.Supp. 33, 38 (D.N.H.1995)). Thus, an expert's opinion that is only connected to the existing data by the bald assertion of the expert is not reliable and should be excluded. See id. The opinions of Messrs. Allen and Hiller should be excluded for at least six reasons. First, Mr. Hiller and Mr. Allen never analyzed SMG's actual lists. See Ex. 8, Allen's Report at 1; Ex. 7, Hiller Dep. at 22-23; Allen Dep. at 48. Instead, Mr. Hiller and Mr. Allen relied exclusively upon data from PrimeNet, which combined SMG's lists with data from various other sources. Ex. 2, Russell Dep. at 68; Ex. 7, Hiller Dep. at 120. In addition to combining SMG's data with other data, PrimeNet manipulated SMG's data through various processes including, removing duplicates, assigning "common names," i.e., names supplied by more than one vendor to the account of one vendor or another, running various postage certifications, applying a "reservation number standard" (so that CPI could track performance of lists supplied by various vendors) and creating a mail file. Ex. 2, Russell Dep. at 19-21. 10

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 11 of 14

Through this process, PrimeNet changed SMG's data; indeed, in some instances, CPI and its experts had no way of knowing what data originated from SMG. Ex. 2, Russell Dep. at 32-33. Without knowledge of precisely what records were provided by SMG as opposed to other list vendors, Mr. Allen and Mr. Hiller cannot reliably analyze SMG's data. Second, CPI has no evidence that the data from PrimeNet is a reliable source from which its experts can analyze the quality or quantity of the names provided by SMG. To the contrary, CPI's own expert, Mr. Hiller, questioned whether the "reservation number standard," one of the manipulations that PrimeNet performed on SMG's data, was being applied correctly to the data. Ex. 7, Hiller Dep. at 23-25. Moreover, CPI has sued PrimeNet, asserting that PrimeNet's work was inadequate and incompetent. Ex. 3, Grace Dep. at 158. CPI has offered no reason for its use of PrimeNet's manipulated data set rather than SMG's actual data, has made no attempt to establish the reliability of the PrimeNet data and will be unable to do so at trial.5 Thus, Mr. Hiller and Mr. Allen's testimony must be excluded because it does not comport with the requirements of Rule 702 that testimony must be based upon "sufficient facts or data" and that the data used is reliable. See The Ninth Avenue Remedial Group, 141 F. Supp. 2d at 960 (holding that in the absence of any evidence of the reliability of the data relied upon by an expert, the expert's testimony is not based upon sufficiently reliable data and must be excluded). Third, PrimeNet apparently assigned certain SMG records, including (1) common records between SMG and ASL and (2) SMG records CPI used for its "creative test mailings," to a "reservation number standard" that does not credit SMG with those records. Ex. 2, Russell Dep. at 32-34, 60-61; Ex. 5, Allen Dep. at 118-22. Thus, any conclusion based on Mr. Allen's

5

There are no witnesses from PrimeNet listed on CPI's witness list. 11

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 12 of 14

analyses undercount "good" names provided by SMG by an unknown amount. Ex. 5, Allen Dep. at 118-22. Fourth, CPI's experts' analysis encompasses only approximately 15% of the records SMG supplied to CPI under the Renewal Agreement. Not only do Mr. Allen and Mr. Hiller use this incomplete data as the basis for their opinions, but they also failed to set forth any reason as to why or how conclusions that they reached upon this fraction of data could be extrapolated to all of the data SMG provided to CPI. Fifth, Mr. Allen used the wrong version of the computer program to perform his CASS certification. To analyze the quality of SMG's records, Mr. Allen performed a "CASS certification" using a software program designed to match addresses on a list with addresses recognized by the U.S. Post Office. The records from the list that do not match an address on file at the U.S. Post Office are designated as "CASS failures." Mr. Allen performed this certification on data supplied to CPI for mailings beginning in January 2004. Ex. 5, Allen Dep. at 42-44. Mr. Allen, however, used a program that was updated several times thereafter to be current as of December 2004. Ex. 5, Allen Dep. at 42-44. This difference in time is extremely relevant because the "CASS certification" program updates who moves and tracks changes in addresses in the U.S. Post Office. Ex. 5, Allen Dep. at 42-44. Thus, to get accurate results regarding the number of "CASS failures" on a data set, the date of the data and the date of the program file should be approximately the same. Mr. Allen concedes that this mismatch generates unknown errors. Ex. 5, Allen Dep. at 42-44. Finally, Mr. Hiller's testimony about SMG's marketing materials is pure ipse dixit. He did no investigation of the underlying facts, applied no discernable methodology and selectively cites only fragments of the marketing brochure that he concedes is not misleading in its entirety. Such testimony is clearly inadmissible. See Mitchell, 165 F.3d at 782 (excluding 12

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 13 of 14

expert testimony where the opinion evidence was connected to the data only by the ipse dixit of the expert). IV. CONCLUSION

For each of the foregoing reasons, SMG respectfully requests that this Court grant SMG's Motion in Limine to Exclude the Expert Testimony of Mr. Nathan Allen and Mr. Thomas Hiller. Dated: June 24, 2005 Respectfully submitted, s/R. Daniel Scheid LEWIS SCHEID LLC R. Daniel Scheid River Point Building 2300 Fifteenth Street, Suite 320 Denver, CO 80202 Telephone: (303) 534-5040 Facsimile: (303) 534-5039 E-mail: [email protected] KIRKPATRICK & LOCKHART NICHOLSON GRAHAM LLP Patrick J. McElhinny, Esquire Dianna S. Karg, Esquire 535 Smithfield Street Henry W. Oliver Building Pittsburgh, PA 15222 Telephone: (412) 355-6500 Facsimile: (412) 355-6501 Counsel for Plaintiff, Student Marketing Group, Inc.

13

Case 1:04-cv-01258-LTB-BNB

Document 90-2

Filed 06/24/2005

Page 14 of 14

CERTIFICATE OF SERVICE I hereby certify that on this 24th day of June, 2005, I electronically filed the foregoing MEMORANDUM IN SUPPORT OF PLAINTIFF STUDENT MARKETING GROUP, INC.'S MOTION IN LIMINE TO EXCLUDE THE EXPERT TESTIMONY OF MR. NATHAN ALLEN AND MR. THOMAS HILLER with the Clerk of the Court using the CM/ECF system which will send notification of such filing to the following e-mail addresses Rosemary Orsini, Esquire Brian Matise, Esquire BURG, SIMPSON, ELDREDGE, HERSH, JARDINE, P.C. [email protected] [email protected]

s/R. Daniel Scheid LEWIS SCHEID LLC R. Daniel Scheid River Point Building 2300 Fifteenth Street, Suite 320 Denver, CO 80202 Telephone: (303) 534-5040 Facsimile: (303) 534-5039 E-mail: [email protected]