| Category | Comment | Answer |
|---|
| Intro/Standards | ESE should recommend that standard products - -. The next sentence addresses user needs, which are often NOT HDF, or NetCDF, but native formats that have been in use by that community. I also note that the balance of the paragraph is ECS-centric, and may not represent the best practices of the industry. To require these is not in the spirit of the SEEDS activity. Many data products are not in HDF, but in native formats. This is not likely to change because the community standards apply. | The phrase "mission standard products" has been changed to "standard products." |
| Chapter 1/LOS | Documentation: The data service provider must also maintain documentation so that is readable in the future. This implies that the documentation is in machine-readable format, and that the format is maintained and readable by current technology. | The Documentation requirements and LOS refer to the creation and content of documentation, while preservation of documentation is covered in the Archive requirements and LOS, see item 2.4.a in Working Paper 5 'DSP Model - Requirements and LOS'. To reinforce the requirement, item 2.4.c will be revised to explicitly include documentation: 'The data service provider shall provide for secure storage of all standard or other science products, including their documentation, ... '. Requiring access to documentation, with the 'machine readable' implication, will be reinforced by changing Access and Distribution item 2.6.a to include documentation explicitly: 'The data service provider shall provide users with access to all data, products, and documentation holdings... ' |
| Processing: The data service provider may also produce algorithms and software for in-house generated data products. In this and in all cases, the software must be documented. | Processing item 2.3 a will be amended by adding to the end 'including complete documentation of all product generation software used by the data service provider'. |
| Chapter 1/LOS | Archive: Provider shall - - take steps to maintain and preserve the data in its archive. The provider must also maintain the format libraries used to generate and read the data. | Archive item 2.4.a will be modified by adding to the end '... including format libraries used to read and write the data and products'. |
| Media preservation: This activity can also be based on retention times, if these are short compared to the media lifetime. | It is assumed that this comment is in reference to Archive item 2.4.j. See next response. |
| Migration: Migrate to new media or new technology. Search and Order: Add ESIP Federation to the list. | Archive item 2.4.j will be changed to read 'The data service provider shall plan and perform periodic migration of archive to new archive media and/or technology, taking into account planned retention times as well as media and/or technology lifetime.' Search and Order item 2.5.e includes Mercury in its list of 'selected external catalog search capabilities', which is the ESIP Federation adopted system. |
| Chapter 1/LOS | Access and Distribution: Data availability within ten seconds is extreme, and not needed nor supported at this time. A more reasonable time might be ten minutes, or one hour. This is a cost driver. | Working Paper 5 Access and Distribution item 2.6.f includes as one level of service a 10 minute product availability time. Designation of this or other level of service as minimum or recommended for a future data service provider will depend on its particular mission responsibilities. It is anticipated that the cost of providing on-line access will continue to fall, and that user demand for on-line access will continue to increase, with the point of cost benefit balance shifting accordingly as time goes on. |
| Read software must be distributed with all data. | Access and Distribution item 2.6.a will be changed by the addition of '(including read software)' following 'documentation'. |
| User support: Staffing levels. No one can afford one USO staff per 100 customers. We have of the order of 5000 users per year, with a staff of 3.5 in USO and about 6 Data Engineers who also serve in this function part time, or an FTE of 6-7. Thus 1 FTE per 1000 is about right. 24 X 7 does not exist in most, if not all, DAACs. We operate on a 8 X 5, as do most. This is adequate because we have a good web page that answers most questions via FAQ pages, and simple interfaces with on-line data access. | In User Support item 2.7.a, the three levels of service will be changed from one user support staff member per 100, 500, and 1,000 active users to 500, 1000, and 2,500 active users. Active users are those who put in requests for data, products, and services, and who might need help in making choices, interpreting formats, etc., as opposed to 'casual' users who might simply access a data service provider website and download a prepackaged product without interacting with the data service provider staff or causing execution of any system process more complicated than a simple download. |
| Chapter 2/ Standards | Missions: We need to add the current ESSP missions, etc. In the community involvement, we need to include the DAAC USOs because they are the agents for the customers. | The list of near-term missions to consider was provided by HQ as missions or projects that were in particular need of guidance so as not to conflict with standards SEEDS would be likely to adopt. The purpose of the near term missions recommendations is to provide recommendations pending the "spin-up" or the Standards Process that will ultimately adopt Enterprise-wide standards. The standards process is intended to involve stakeholders throughout the Enterprise. DAAC User Services Organization members were contacted to acquire lessons learned experience with standards for near-term heritage missions. For example, EDC DAAC USO was contacted to acquire lesson learned with data distribution standards for Landsat-7 data and JPL DAAC USO was contacted to acquire lessons learned experience with Janson-1 and Topex/Poseidon data and metadata standards. |
| Chapter 2/ Standards | # 5: may be different . . .. This is not true for many of the legacy data sets which are in native format, and some of the HDF formatted data. | The commenter clarified that this comment refers to recommendations on interchange formats in section 2.2.5. He thinks that we should recommend characteristics that interchange formats should have, such as self-explanatory, completeness, expressive power, and etc, rather than specific interchange formats. The intention of our recommendations for near-term missions is to provide concrete and specific recommendations for the "near"-term that are compatible with existing systems from which we are evolving. |
| Chapter 3/ Cost Model | Software previously funded by NASA. Note that there is a lot of good software not funded by NASA, and many of us use this. | In Section 3.4, the final item under 'General Requirements' will be modified by the addition at its end of 'as well as software from other sources.' |
| Production, Archive and Distribution: Next to last bullet implies that if the LTA does not pick up before end of mission, the data is dropped. This is not, nor should it be, current policy. NOAA has just defaulted on the LTA for now because of budget issues. We will maintain our legacy data until the LTA is in place and operational, and accepts the data. | In Section 3.4, the fifth item under 'Production, Archive, and Distribution Requirements' will be modified to read: 'All standard data products will be archived until transfer to an approved permanent archive'. |
| Chapter 3/ Cost Model | One can include commercial data in the cost modeling activity by making all data anonymous, and by sharing the results of the model with industry. The hotel industry contributes cost data to a modeling activity that benefits the industry, under the condition that no-one can determine who is responsible for what data. This model includes operational costs which are far more sensitive than our cost data. | The information in the Comparables Database (CDB) that would be distributed as part of the Cost Estimation Tool package does not contain explicit identification of the sources of the information, i.e. the identities of the data activities included in the CDB. The inclusion of commercial data activities will be pursued in the future if one or more are identified and seen as comparable to ESE data activities and when the privacy features of the CDB can be demonstrated to them. |
| Chapter 4/Life Cycle | It should NOT be an option whether data is submitted to the Long Term Archive. This must be mandatory. The Long Term Archive will prioritize the data holdings based on budget and advice from the community it serves. Some holdings, such as Level 0 are not optional in any case. 4.3: There must be requirements on data access in the future. This implies Stewardship and preservation standards, and metadata and technology refresh standards. | Agree that the transfer of the data to an LTA is not an option for a mission. However, as you state, it will depend on the communities needs, budgets, etc. Will modify "Purpose" section to reflect this. Also, envision access will be addressed as a "level of service" requirement. |
| Chapter 4/Life Cycle | There must also be data stewardship and maintenance standards. | Agree and we believe we address this issue in the first paragraph of Section 4.4.5. |
| Chapter 4/Life Cycle | Missions can end without the data going to a Long Term Archive, because we do not have this in place today. We use an "intermediate" archive which is the Active Archive until transfer can be arranged; sometimes years later as in the case of NOAA LTA. Note that keeping such items as drawings and instrument specifications, while essential, can lead to distribution issues because of ITAR regulations. These are fundamental, and MUST be observed. The Data Management Plan should cover all the items listed in this section. | Agree. The data management planning that was introduced in the "Purpose" section must address the entire transition process, which could include an "intermediate archive". |
| Chapter 4/Life Cycle | While a hard copy is OK, documentation must be maintained in electronic form in current technology (Word, PDF, . . .) | For future missions, we agree that softcopy is preferred and probably a must. However, for older missions it may not be feasible to convert all documentation to softcopy. |
| Chapter 4/Life Cycle | The LTA must retain all data products that the community needs unless the capability exists to reprocess the data from the LTA. In most cases this capability will not exist because the software has not been maintained in current technology, and cannot be used without a large effort, thus rendering the data effectively unavailable. | We believe that we have stated this within combinations of several bullets in section 4.4.5. We also agree that it will be a hugh effort to keep the algorithms functional several years into the future. The concept here is to have multiple levels-of-service. These levels-of-service can and will be different depending upon the uniqueness of each mission. In some cases higher-level products will be archived. In other cases they may be created. |
| Chapter 4/Life Cycle | No records should be stored only in paper form. These should be available in electronic form through the LTA. | We agree in an ideal world. One of the biggest problems is dealing with the various types of documents such as design drawings and other unique notes that people may have never put into softcopy. |
| Technology Infusion | The principal reason for the failure to infuse new technology into ongoing operations is that technology development has been given to technology developers. If this is to work, then the development must be led by the organizations that need the technology, or implementation by an operational activity must be a significant part of the development proposal. This has not been the case in the past. Too often, technology has been developed without an end user in the loop, only to find that the technology is inappropriate, too difficult to implement, too expensive to operate, etc. That is technology that will not be used. The Applications Division requires that for an application to be funded, the end user must sign up to use the application in the proposal, and in fact must lead the proposed activity. This should be applied to technology proposals. There is an argument for the development of technology for the sake of technology, but it should not be confused with technology that will be used by an operational activity. | Added the following to the list of problems: "New technologies may have been developed in isolation from real-world user and operational requirements". Extended the recommendation to include the following: "The recommended technology infusion process should integrate fully with existing technology development processes for maximum effectiveness. For example, matching technology developers with potential early adopters should not wait until a technology is fully developed, but instead should be done as early in the development process as possible to ensure technologies are aligned with real-world requirements."" |
| Chapter 7/MPAR | One critical function is the control and allocation of resources. The Program Office must oversee the initial allocation of resources. These may be passed down, with requirements, to lower levels, but the Program Office has the responsibility. | p61. We agree, however the issue of program office resources will be discussed in another section, probably governance. Please note p 72, point 4, states: "The funding flow from the SEEDS Program Office indicated here covers only the activities needed to meet requirements uniquely arising from SEEDS." |
| Chapter 7/MPAR | Delete. This is the last sentence of the previous paragraph. | Done. |
| Chapter 2/ Standards | Suggest you define what an "ESE Standard Data Product" or "ESE Mission Standard Data Product" is. The use of standards facilitates interuse of multiple product types in Earth system science studies, so standards should apply to products for that purpose whether or not they are associated with a "mission" per se. | The near term missions standards recommendations are for the named missions and this may have colored our particular terminology. We did not intend to introduce a distinction between "mission standard products" and "standard products". We will edit the document to change "mission standard" to "standard". Standard products are different from and should be held to higher standards than "experimental" or "research" products. |
| Chapter 2/ Standards | A frequent criticism has been that products are not offered in simple native or binary formats that are easier for users to use. Suggest that DSPs be given the latitude to offer a simple alternative where the users ask for it. | Agreed. The near-term mission standards study recommends that "ESE distribution components must enable packaging of standard products in formats and ways that emphasize end-user needs and convenience (Section 2.2.5.1, No. 7)". |
| Chapter 4/Life Cycle | Add a recommendation for establishment of a data life cycle or "stewardship" working group, that would recommend standard practices for data protection and preservation, standards for documentation, especially content, to ensure long term usability. The long-term value of data can be compromised if not destroyed by inadequate documentation (broadly defined - see the 1999 Global Change Science Requirements for LTA for scope). The uncertainty about LTA is no reason not to give these topics priority - if data is not cared for and information about it not collected and secured in the near term, it will not be there for the LTA, and the LTA will not be able to recover it. A number of the specific recommendations in section 4.4.1 talk about documentation that should be provided with data and products, and that an archive accepting products from a project assumes responsibility, but this can only work if there is an agreed standard that the project must meet, or that the archive can withhold acceptance until it is met. | Based on this comment and others received at the SEEDS Workshop, we will add a recommendation to establish a "data life cycle" working group. Our original intent was to get data life cycle requirements and issues into the discussions of the other working groups that have been defined but agree that a dedicated working group does add focus and visibility to these issues. Membership and funding for such a working group will have to remain TBD for the moment." |
| Chapter 4/Life Cycle | Add a recommendation that a process be established for science assessment of long term potential value of data and products for climate research, etc., so that in a resource constrained environment science priorities are considered when decisions are made as to which data and products are retained, which receive a maximum effort at documentation and preservation, etc. | Agree, and will add appropriate language. We will also need to review existing science advisory bodies to see if an owner for such a process already exists. |
| Chapter 4/Life Cycle | What does this mean? | Agree this is confusing. Point is that data life cycle planning must be addressed at the mission proposal stage. Will modify bullet accordingly. |
| Chapter 4/Life Cycle | This should say that NARA policies / rules for data disposition need to be followed, or if there's a separate point then suggest adding that as a bullet. | Point is that NARA can be considered as a partner in data life cycle planning. Will reword bullet to be clearer and to include adherence to NARA rules and policies. |
| Chapter 4/Life Cycle | Second sentence says "The Active Archive's major role is to serve the day to day needs of the mission." This seems to ignore a major role in providing data and products and supporting services to the scientists / applicationers who are carrying out the ESE science and applications program, the whole point of having the data, and secondarily the broader science community. This paragraph makes it seem like you have an "active archive" imbedded within a mission serving the immediate mission science team, then handing off to the LTA. If you mean for Section 4.4.4 to only address immediate mission support, then add a new Section 4.4.4 and one half that addresses support for the ESE science and applications program, as the DAACs do today. This seems like a very large hole! | We do believe that the Active Archive's primary role is to serve as the archive while data products are being processed and reprocessed to the highest scientific quality possible using on-going data validation and quality assessment results. While this is occurring, the data are also of use and being provided to the broader science and application community of users. Both of these functions are provided by the DAACs and both are important to the missions and the ESE. For this document, we will expand the Active Archive role to include the additional functions. |
| Chapter 4/Life Cycle | This section is missing a fundamental tie back to the ESE science and applications program (discussed in the item above, and also visible in section 4.1 where not only is no mention made of the ESE science and applications program, but the only focus is on 'mission science teams to LTA' as if that defines the data and information services world. In addition to the huge missing link noted above, the whole plan for handling data and products should not be to schlep off to an LTA as soon as possible, but rather to develop a life cycle plan that takes into account the needs of the ESE science program and paces the migration of data according to its needs. Some products used actively in climate change research may be have to be kept readily accessible in a NASA data services provider for many years, for use in building long time series, perhaps involving intercalibration with other products from other sensors, perhaps needing supporting services to facilitate interuse, etc., as needed to support the ESE science program. | We will expand the functions of the Active Archive to reflect the additional functions as noted above. However, an implication in the comment is that as long as data is useful for climate change research it must be retained in a NASA data services provider. This implies further that data in an LTA will not be readily accessible and that only data that is beyond its main utility will ever be transferred to an LTA. At this point in our analysis, we are not prepared to make that statement. |
| Chapter 5/Reuse | Suggest you drop the obscure terms of art "mission critical" and "mission success". Figure out what the real distinction is that you are trying to draw, and describe in those terms." | It is difficult to capture the essence of each environment in two words. However, our interaction with various community representatives indicates that the Mission-Critical and Mission-Success terms are resonating well with the community, and seem to reflect the real distinction between the two environments. In the absence of proposed alternative wording, we are closing out this comment and will make no changes at this point, and the terms and definitions will stand for now. When and if a SEEDS program office forms and the working groups are formed, we believe the stakeholders themselves can adjust their respective charters once they are fully engaged in the process. At that point, they may well choose to rename themselves. |
| Chapter 9/Next Steps | It says that "the term DSP is used for any entity responsible for meeting a set of requirements under a funding arrangement with NASA." This is too broad - clarify by changing to "any entity responsible for meeting a set of requirements for providing data and information services under a funding arrangement.". That's what the words "Data Service Provider" say, yes? In the next sentence, you have the ECS Contractor listed as an example of a Data Service Provider - plainly wrong, they are a system vendor. Should suppliers of hardware and software, or reports and engineering studies like SGT, anyone who meets any requirements, be a DSP? I suggest that the DAAC User Services Working Group is an example of excellent coordination among DSP's with ESDIS oversight and assistance that has been going on for years, and that under SEEDS this should continue, be recognized as a SEEDS type, activity and add participation from other DSPs who also provide services to users beyond an immediate science or project team. | p72, point 1. We agree. We will revise text to clarify the definition of a DSP, and correct the intent of 'ECS Contractor'. Implementing a SEEDS version of the DAAC USWG is interesting recommendation and will be taken under advisement. |
| General/ Text edits | Cost Estimation, Findings, better to say "produces estimates" than "produces data". 2. p. 35, 2nd pgh, the reference to "Section 7.5" should be deleted. 3. p. 39, Next Steps, change "comparisons database" to "comparables database"; the database contains data on comparable activities, not comparisons. | Changes have been made. |
| General | Detailed comments will have to wait for next week, but so far these documents seem to suggest that there has been very little examination of lessons learned from the ESIP Federation. None of the teams, including ours, really mention the Federation. The whole concept of federalism and the reasons to choose federalism are absent. Note that the instigating Handy report recommends federalism specifically, not just any form of federation. I will suggest text for the documents next week, but to do it properly would require more than the hour I can donate. It is Appendix F in the Freiman NRC Report that has the detail (http://www.gcrio.org/USGCRP/LaJolla/appF.html). Though peer review is mentioned, the recommendation goes well beyond simply better peer review: we must now embrace a revolutionary expansion of the conceptual model that governs the management and operation of the [EOS] system by affording the scientific community full partnership with shared responsibility. To effect this recommendation, it will be necessary to examine the systems implications of reconfiguring EOSDIS as a loosely- coupled federation of quasi-autonomous partner organizations | Federalism is a governance issue that is still being formulated. The Formulation Team is sensitive to and has responded to lessons learned. Please refer to Section A.4, Lessons Learned, from the NASA HQ NewDISS Strategy Team Report. The Formulation Team would appreciate receiving any ESIP Federation 'lessons learned' documentation that can be considered in SEEDS formulation. Peer review is obviously an important process in NASA's ESE, but it is currently not a topic of discussion for the SEEDS FT. You are encouraged to submit comments and recommendations on any changes to the peer review process to NASA HQ, Code Y. The recent NASA CAN, and SEEDS requirements both within the CAN and in formulation development, have addressed many of the actions listed in the Freiman Report's Recommendation 2 and the points listed in this comment. |
| General/Standards? | In a separate issue, not mentioning RDF or the semantic web in Volume I may be like not mentioning the internet in 1987. These standards bring the concept of data to the World Wide Web (http://www.w3.org/2001/sw/ ) and set the stage for concept-based science communication. At least the long-term standards team might consider mentioning it. Also, the mentioned standard called OAIS, which is a standard for describing the structure of archived digital material, doesn't t seem to be related to OAI from the Open Archives Initiative (http://www.openarchives.org/). This latter standard is widely used in the digital library community and is being used in the NSF-initiated National Science Digital Library (http://about.nsdl.org/ ). In one of our few resolutions, the ESIP Federation voted to work closely with NSDL, so it might be appropriate to mention it in these documents and discuss the relation between it and SEEDS. | Near-term recommendations concentrate on the near-term and are somewhat conservative. The standards process addresses a process for community input without suggesting the resulting standard. The community in future will decide which of these standards are appropriate to be included. |
| General/governance? | We concur that a centralized SEEDS program office is a good mechanism to maintain and update statistics on usage of the data and products. Such work is more readily accomplished at the SEEDS office rather than the provider level. The provider could work with the SEEDS office to point out areas of surveillance or keywords which indicate usage of the data. The SEEDS office could then report back to the office (quarterly?) on new endeavors which use the provider's data. The provider would submit as part of their usual quarterly or monthly report any interest or queries made directly to the providers office. | We agree. The comment on having the DSPs point out areas of surveillance or keywords which indicate usage of data is very pertinent and one that will be pursued in the Working Group. |
| Chapter 7/MPAR | Metrics planning and reporting (P. 67): We can track the number of citations of the prime reference article on a dataset. This is easily accomplished using Science Citation Index. This would provide a measure of community interest and number of users of an ESE-sponsored dataset. | Good comment. We will look into SCI as a metrics tool. |
| Chapter 7/MPAR | Optimal estimation retrievals report their own diagnostics about algorithm performance, reliance on a priori data, convergence success, and degree of fit of results to observations... These would provide additional metrics to a user of the data about the amount of error expected in the results. An illustration of the use of these metrics in a water vapor profile retrieval is given in: Engelen, R. J., and G. L. Stephens, 1999: Characterization of Water Vapour Retrievals from TOVS/HIRS and SSM/T-2 Measurements. Quart. J. Roy. Met Soc., 125/553, 331-351. | We agree and assume the comment refers to metadata provided by data set developers and providers. But we have addressed this comment indirectly -- please refer to the report's accountability recommendations on pp 64-65. Three levels of product quality are defined with sample metrics. |
| Introduction/LOS | Comments: Breaking the requirements into minimum and recommended is a useful way to proceed; provides guidance without over-regulation. | In practice, the 'minimum' and 'recommended' levels will vary from case to case depending on the mission of each data activity. In Working Paper 5, 'DSP Model - Requirements and LOS', a number of levels are defined without reference to 'minimum' or 'recommended', but an indication of how 'minimum' and 'recommended' levels might be selected can be found in Working Paper 6 'ESE Logical DSP Types', section 4. |
| Am surprised that "operational" processing is either 30 days (minimum) or 2 days (recommended). Some NASA satellites provide data that can and should be used within hours, and great strides are being made on a number of fronts in this regard. Consider introducing another data category, such "expedited" with much shorter recommended and minimum...otherwise there will be data products that simply do not fit the present categories." | A new requirement / level of service definition will be added as Processing item 2.2.c in Working Paper 5: 'The data service provider shall provide near real-time processing of selected product types. Levels of Service: 'Near real-time products shall be generated within 30 minutes of ingest/availability of required inputs; Near real-time products shall be generated within 3 hours of ingest/availability of required inputs; Near real-time products shall be generated within 24 hours of ingest/availability of required inputs.' A note will be added to the second paragraph of section 1.4: 'The requirements and levels of service provide a general framework for describing ESE data activities. When requirements are written for a particular new data activity, requirements such as for near real-time processing at a certain level of service will be used or not as appropriate for its particular mission.' |
| Archiving is a difficult issue. What is secure archiving? The answer is different if the issue is accessibility a century from now (appropriate for some data) or if, instead, the data will have little value after analysis by contemporary users. The SEEDs recommendations are appropriate for business as usual...I wonder if there needs to be room for special consideration for data whose very long term preservation is justified. An example from our work: we produce hundreds of MODIS--derived maps of rare flood events. We provide these maps in digital form over the internet and can provide hard copy as well. The more decades that pass after the occurrence of such an event, the more critical it is that the remote sensing evidence be preserved. There are ecological data sets that also fit this category. Even major government archives are not necessarily secure storage; funding for such can be lost. | Working Paper 5 Documentation and Archive requirements include levels of service appropriate for long term archiving, and Working Paper 6 ('ESE Logical Data Service Provider Types') section 4 explicitly addresses the requirements and levels of service for a long term archive. Reference is made to relevant standards, e.g. for documentation appropriate for long term archive, and work on such standards is urgently needed but is outside the scope of this study. |
| Perhaps SEEDs planning should include special provision for physical dispersal of media archiving such data as "publications" held by university libraries? | Your suggestion will be referred to the SEEDS Data Life Cycle Team for consideration. |
| Regarding distribution and shipping, again, network delivery within 24 hours could be completely inadequate rather than a useful minimum standard depending on the data set. A new "expedited" or "real time" data category might be needed. Also, it might be better to avoid specifying days, seconds, etc as in the table. Distribution and shipping are critical. It MIGHT be better to define these as appropriate to identified user needs, measured via a post-delivery query. | Adopting a definition 'appropriate to user needs, measured by a post delivery query' would not work in advance, i.e. for a procurement of a new service, nor could a requirement written that way provide a basis for making an estimate of the costs to meet it The intent of the quantitative requirements is to reflect user needs, as well as support cost estimating, and a key task of the future Level of Service Working Group will be to update and revise the requirements and levels of service to ensure that they do reflect user needs. |
| There seems little purpose in setting as the minimum level "access to limited groups of scientists or applications users". Specialized science data provider should be under no obligation to provide user support to public, but neither should data provider be allowed to restrict access by public. Unless, as is often true (e.g. Radarsat) data distribution must be restricted per international agreements. So Minimum might read: "Access to the public without advertisement or support, unless data distribution restrictions apply" and Recommended: "Public access to all users unless data distribution restrictions apply". | In Working Paper 5 Search and Order items 2.5.1 and Access and Distribution 2.6.1, the second and third levels of service will be modified to 'access to the science and applications community, with at least a minimal capability for public access', and 'access to a limited team of scientists or applications specialists, with at least a minimal capability for access to the science and applications community and the public'. A note will be added to both items: 'Public access may be limited in some cases by constraints levied by the original data source. Data service providers with a highly focused primary mission (such as a flight project instrument team) may meet the requirement for public access by teaming with other data service providers actively engaged in public access and services. |
| The staffing levels for user support may be too rigid. 1000 active users of one type of data may need little support, but a different data set may require individual guidance of individual but important end users. Avoid this kind of metric, which will just cause problems. Also avoid too much dependence on survey (user satisfaction) results, which have their own problems. There must be another way. Possibly this would involve record keeping by user support staff: so recommended might read something like "staff sufficient to respond immediately to user queries 9 days/10" and minimum: "staff sufficient to respond within one work day to user queries 9 days/10." | The LOS Working Group will be asked to consider possible alternatives to the present level of service definition, which is being adjusted per information received about actual levels of user support staff per number of active users. The goal is a quantitative definition that captures the genuine requirement and provides a reasonable basis for estimating user support staffing needs. |
| Again, to specify that a help desk be staffed 5 days/week is too rigid and can cause unneeded costs. As in industry, user support can be provided by individuals with flexible work hours. As per the above, the emphasis should be on how efficiently users are actually being serviced, and with allowance for some intervals being very busy and others slack. | The specification of a help desk being available 5 days per week, eight hours per day (for example) does not require that a single person be assigned that one task, only that a user can reach some person for help on a five by eight basis. How the help desk function is operated and staffed is up to the data activity; an activity might have a group of people engaged in other work who contribute part of their time to the help desk. But the quantitative requirement does provide a rough basis for planning by a data activity and for cost estimation. |
| Outreach and training: the benchmark for recommended could be set a bit higher. For example, one important task that needs to be encouraged: interfacing with software vendors so, for example, they read new file formats. | The extension of 'outreach' to include explicit contact with software tool vendors to encourage them to enable their software to work with a data activity's products seems more reasonably an ESE or SEEDS level activity, and will be considered as an activity of the SEEDS standards working group. |
| Introduction/ Standards | Comments: Most of the text is appropriate and provides good base for a working group. However, to be avoided at any cost (!) is requirement of specific formats...rather than endorsement or encouragement of certain formats, which is appropriate. ESE has shot itself in the foot often in this regard. We want wide data dissemination and use, but provide data in ever-changing formats that commercial vendors are unable to keep up with. Worse, we then insist that these non-standard formats are the only ones allowed! The best words in the existing text are: "Encourage adoption of existing successful standards. Develop new standard if no existing viable candidates. " That is a sensible approach. | Our intent is that usable, well-tested, and appropriate standards are chosen based on input from the users affected by the standards. The approach is to provide a structure and process to allow this, rather than to mandate top-down standards adoption. |
| Introduction/Cost Model | Comments: Most of the extended text is reasonable and needed. If frustrating! Some centers or activities will turn out to be "more expensive" than others, but due to unalterable circumstances. NASA-ESE needs better information, but very significant money and time will be needed to produce the needed comparable costs for data service providers. And these will likely remain subject to dispute. And ESE can end up still needing to continue to support providers and activities that cost out higher per unit deliverable than others... This is a critical activity that needs very careful thought. Avoid adding to costs too much by insisting on accounting overkill. Avoid pitting needed but high overhead data providing facilities such as JPL against lower overhead universities. | Your points are noted we agree in principle, but they do not generate changes to the document. |
| Structure the cost model to identify inefficiencies...to encourage cost reduction. | Identification of inefficiencies and thus opportunities for cost reduction implies analysis of information describing existing data activities, i.e. the Comparables Database (CDB), to identify (comparative) inefficiencies at existing activities. This is not a purpose of the CDB. The success of the CDB as a basis for cost estimation depends on the accuracy of the information provided by the CDB data activities. It is a burden on data activities to provide input to the CDB, and knowing that a purpose of the CDB was to enable management to make critical comparisons would have an adverse impact on their cooperation. Separate management processes already exist for oversight of the performance and efficiency of ESE sponsored data activities. |
| Introduction/Life Cycle | Comments: As per above, some data sets are so valuable that long term electronic archives are not enough. Can life-cycle categories be defined to include such? Increasingly, NASA management and science P.I. and applications end users are going to need to jointly and carefully identify which data setsmust be earmarked for permanent preservation and accessibility, and with an eye to the fickle nature of the federal budget. There should also be mechanisms to provide economical and "permanent storage" for some data sets without easy access: so as an intermediate stepbetween losing the data and providing for long term support. Am not sure about point-of-contact text. Such is useful, but not essential...adequate documentation and metadata ARE essential. Re: "The data provider should also provide a comprehensive training course in the science and prior management behind the instrument, data, processing histories, format(s), and algorithms". That is a tall order! And not specific enough...hard to say exactly what this "course" should be. | We believe that the new recommendation for a Data Life Cycle Working Group will be able to address many of your points by defining the process for making decisions that govern the transition of data through its life cycle. Certainly desired levels of service and budget considerations must be part of the process. |
| Introduction/Reuse | Comment: The above are excellent and sensible recommendations. | Thank you. |
| Introduction/ Technology Infusion | Comment: Text does not appear to say who will do this. There are almost no examples of what this entire chapter is about. Much needed: some specifics. Even what "technology" refers to, exactly, is not spelled out (unless I missed it). Attempts to envision the future are difficult. Why not focus on incorporation of presently underutilized innovations? | Added the following to the description of the recommended activities "... that are managed by the SEEDS Program Office and performed by the ESE community." Extended the description of technology infusion projects as follows: "... projects would be focused directly on deploying < new or underutilized > technologies into an operational system." |
| Introduction/MPAR | Comments: I must not be the only individual who will have trouble with the disconnect expressed here between the extensive chapter on cost models...and this one on metrics. Is it sensible to separate these two ways of measuring performance? There needs to be a bridge between the two. It may already be there but I missed it. I strongly concur that, where appropriate, quantitative assessments of data service providers be made, but quantification (developing metrics) cannot entirely take the place of NASA ESE professional and qualitative assessment of data services. Many metrics, such as number of scenes ordered, emphasize well-established data sets. What about new kinds of information for which the user community is still small? I am worried about an Office of Metrics! It might have the harmful result of separating explicitly quantitative evaluation of projects from the needed combined quantitative and qualitative evaluation. I am worried about a possible conflict of such an office with rigorous efforts to assess cost-effectiveness in another. But there does appear to be a valid need to measure how well especially the larger data service providers really are serving the public, and how busy they are. And there will have to be hard decisions made concerning what metrics to use... | (1) The bridge is between levels of service and cost estimation, not cost estimation and metrics. Levels of service are performance requirements that provide the framework for the cost estimation model. The bridge between levels of service and metrics is the degree of accountability for a ESE-funded DSP. Please refer to Levels of Accountability, p 64 in the draft Report. We have revised the text to clear up the concerns in this comment. (2) We believe new types of information from emerging communities is important and we expect these communities to work with SEEDS in defining appropriate metrics. (3) A SEEDS Metrics Working Group will be formed with community involvement to define metrics, and metrics collection and reporting. Also, we stated at the 3rd SEEDS Public Workshop that metrics will not be used for data service provider inter-comparison. The collected metrics are intended to characterize a DSP's performance and to provide a means to identify and publicize successful accomplishments. A number of ideas for publicizing accomplishments are discussed in section 7.6.3 at the end of the first "finding". |
| Introduction/ Governance? | Coordination wording will lead to trouble. Recommendations, if they are to have any effect, must be implemented. A SEEDS office to implement these recommendations does appear to be appropriate. This report is as yet unclear as to how many working groups would be established, and how many SEEDS-related offices. These draft recommendations need a final, thoughtful, summarizing section. Can cost models and performance metrics be combined? Can the number of working groups be reduced? How to tackle the needs here identified, given limited human resources? What size of office staff would be needed? | We agree. The issue of the program office, its organization, responsibilities, location, and size are all TBD. As currently envisioned, we will not combine the cost model and metrics since these groups address different issues. We do recognize the connection between cost model and the levels of accountability indicated in the metrics section of the report, and that there does need to be communication between the cost modeling group and the metrics group. |
| General | I searched through the document and did not see any references to data grids or persistent archive technologies for the management and preservation of data. I would like to point out that data grids provide many of the mechanisms that are needed to manage distributed data collections. The key concepts are: - logical name space that can be used to apply global, persistent identifiers to digital entities (sensor data and derived data products). The logical name space provides a way to name digital entities that is location independent. - management of distributed state information that is used to support grid services. The distributed state information is mapped to the logical name space for each digital entity. Each grid service maps additional state information onto the logical name space. Examples are replication, aggregation in containers, assignment of descriptive metadata, ... - management of consistency through the application of constraints on the mappings. This is critical when long term preservation is desired. The long term preservation assumes that the digital entities can continue to be discovered and accessed by their original logical names. It is very interesting to note that persistent archives, which manage the evolution of storage technology, are being implemented through data grid technology.I believe the report needs to address these above technical requirements, and point out that these technologies are already being deployed across NASA sites (IPG, ADG, NASA Langley, ESIPs). The reference to the Storage Resource Broker confuses the use of data grids for the management of collections, with the ability to manage digital ontologies, that describe the relationships embedded in digital entities. We would like to use technology that is able to differentiate between the relationships present within a digital entity (structures, semantic tags, time stamps) from the operations that are applied to the relationships. The HDF system tries to do both. The digital ontology imposes an organization on the relationships, such as the order in which they should be applied. A similar concept is used to create a grid ontology to describe the constraints that are applied to grid services to maintain consistent state information. | The technology infusion section addresses process and capabilities. Identifying applicable technologies will be the next step in the process. Added the following to the detailed study report under needed capabilities: "... better data management through global namespaces and location/format transparency..." Added Global Grid Forum to "industry led endeavors such as" the OGC, GGF, etc., under Purpose. |
| Chapter 1/LOS | We can include reliability, accessibility, etc. in a subtopic, such as quality of service, and also make it available to metrics and reporting. | Specific quantitative requirements for reliability or accessibility (that can, for example, drive hardware redundancy and maintenance approach) can be included in a procurement specification for operational services where they are needed, i.e. for time critical handling of downlink data flows or environmental hazards decision support systems. Such would be a special case; almost all ESE data activities will not have such stringent requirements, and can be characterized by the qualitative levels of service used in Working Paper 5 'DSP Model - Requirements and Levels of Service', e.g. Sustaining Engineering 2.9.a, or Engineering Support 2.10.a. In the future, the CET will address such special cases by using system performance measures such as reliability, maintainability, and operational availability. |
| LOS-CET | When using a tool like CET, must assure that the comparison data being entered is accurate and consistent. That said, who or what organization determines that the project is a good representative (one worthy of being an example)? ? | The SEEDS Formulation Team currently and Levels of Service Team working with the SEEDS Office in the future will have the ultimate responsibility for the selection of appropriate data activities for the Comparables Database (CDB). The study team recommends data activities representing a cross-section of the various ESE data providers (the CDB currently includes, ESIP-1, -2, and -3, SIPS, DAACs, and flight project data systems). The study team does its best to obtain complete and accurate information from the activities approved by the Formulation Team. Information received from data activities is examined for inconsistencies and any other problems which are resolved in discussions with the data activities. Data activities that are 'outliers' for some reason (such as having had to cope with an extended launch delay) are excluded as unrepresentative. The CET itself, when producing an estimate, uses those CDB data activities that resemble the new activity being estimated, i.e. it excludes relative outliers. The data activities are made aware that their information is confidential; their original input is mapped to the common base used by the CDB, and they are not identified in the CDB. |
| I'm concerned that past inefficient projects and their costs will perpetuate based on this historical data. | Data activities that are too old or that experienced unusual problems (such as a lengthy launch delay) that would appear as 'inefficiencies' are being kept out of the CDB. In some cases activities are providing estimates of replacement costs for current technology to use in place of original costs of years-old technology. Staffing levels are a major life cycle cost driver. Data activities that were originally less highly automated but which have refreshed their technology and achieved a higher degree of automation are being described in the CDB in their current form. The CET screens and weights CDB activity data function by function in the course of producing its estimate, placing heavier weight on the most recently implemented CDB data activities is being considered. Finally, as the CDB develops, information about new data activities will be added and extended, and information about older activities can be rotated out as appropriate. |
| Does the CET use Moore's Law in estimating H/W and storage costs | The CET will use published Moore's Law type cost curves for projecting future costs for processing and storage hardware, etc. |
| Chapter 1/LOS | The requirements are written as functional requirements. NASA performs performance based contracting and thus is supposed to utilize performance requirements. The functional requirements need to be eliminated. For example, there are facility requirements for backup. The performance requirement is for backup that will survive in the event of primary facility destruction. Then the contractor can decide on the engineering implementation. | The key point is that the requirements in an actual procurement specification should not dictate an implementation approach. While the SEEDS requirements and levels of service are a framework that can be drawn from in the preparation of a procurement specification, they are not a procurement specification as written. They are intended to support an analysis of the functions needed by a data activity and to support life cycle cost estimation, and include some specifics (such as the backup requirements noted) that are needed for that purpose that might not appear in a procurement specification. |
| LOS/Cost | Cost model is not based on reliability--all gov't models I've seen or heard about base the system performance on reliability--This drives the total life cycle cost. Maintenance can be 60-70% of the life cycle cost. | Specific quantitative requirements for reliability or accessibility (that can, for example, drive hardware redundancy and maintenance approach) can be included in a procurement specification for operational services where they are needed, i.e. for time critical handling of downlink data flows or environmental hazards decision support systems. Such would be a special case; almost all ESE data activities will not have such stringent requirements, and can be characterized by the qualitative levels of service used in Working Paper 5 'DSP Model - Requirements and Levels of Service', e.g. Sustaining Engineering 2.9.a, or Engineering Support 2.10.a. In the future, the CET will address such special cases by using system performance measures such as reliability, maintainability, and operational availability. |
| LOS/Cost | Warranty costs/spares costs/inflation cost/parts obsolescence/ training/travel/maintenance costs--these are not clearly identified in your presentation. How does this compare to a cost analysis strategy assessment (CASA)--developed by DSMC. | We are not familiar with CASA, however it does seem to be used primarily by DoD for large-scale acquisition. Most of the items mentioned are handled explicitly in CET - there's a limit to how finely grained it makes sense for the CET to be. We are now adding travel and training based on feedback, and we will address other items (warranty costs, spares, etc.) as time permits. |
| LOS/Cost | Sensitivity or risk analysis--Provide an easy method to vary "critical" inputs to use as sensitivity/risk analysis. Tie in a little probability and automate it to provide a decision support tool. | Sensitivity analysis is being used in the course of development of the CET and the Comparables Database. Adding an ability for the CET user to vary individual selected inputs to see their impact on the output will be considered. As it stands, the CET allows the user to do a 'what-if' exercise after producing an estimate by looping back to a selected functional area or areas to make changes to inputs and re-run the estimate. The ability to have the CET introduce a probabilistic range of variation in its input and produce an estimated range of costs will be considered after the basic model is operational. Ways of introducing into the cost estimation a measure of the variability of the data in the comparables database to produce a range rather than single valued output will be explored, as will be ways of showing the sensitivity of the model output to variation of user provided inputs. |
| LOS/Cost | Other uses of model: LCC estimates, trade off analysis, repair level analysis, warranty analysis, spares/provisioning, risk analysis, resource projection, technology refresh, others. Also, provide geographical representation of numbers! | Other uses for the CET, along the lines indicated, will be explored, within the limits not only of the CET itself but also the Comparables Database. Beginning with the Working Prototype CET, graphics are being added to the CET. |
| LOS | I think adding a sentence or two to the Draft Recommendations Report addressing how the appropriate levels of will be matched to a particular TSP would be useful. Will this determination be a part of the RFP process or negotiated as part of funding negotiations after an award, for example. | The following paragraph will be added to section 1.5 of the Recommendations Report: 'The requirements and levels of service described in Working Paper 5 'DSP Model - Requirements and LOS' provide a general framework embracing a full range of ESE Data Service Providers. They do not constitute a procurement specification for any specific Data Service Provider. When a procurement action (such as an RFP or a CAN) that will lead to the selection of a new Data Service Provider is planned, requirements and levels of service appropriate for its particular mission will be developed, i.e. drawn from the general framework and tailored, added to, etc., as needed in the preparation of a procurement specification. The mapping of the general set of requirements and levels of service to various ESE logical data service provider types shown in section 4 of Working Paper 6 'ESE Logical DSP Types' provides an indication of how this may be done. It can be seen that requirements that apply to a Data Service Provider depend on its type or general mission, i.e. the type of work it does, and the selection of a particular level of service as 'minimum' for a Data Service Provider also depends on its mission.' |
| Not specified | 1. Is there a process in development or already set up for coordinating among SEEDS working groups to: Avoid duplication of effort by different groups. Understand the relationships between/among working groups' tasks, responsibilities, scope, etc. Assure common end goals for SEEDS success. 2. The SEEDS policies, protocols, guidelines, plans, etc. should be a living document; i.e., kept up to date through periodic reviews and updates. | Our intent is to address this through the implementation plan (including how often documents are refreshed and how often groups meet together). |
| PAN | Examine the need for a SEEDS IT security study team. Must balance: 1. Federal requirements vs. university or commercial security. 2. Security vs. access. 3. Funding vs. FTE allocated (i.e., unfunded mandates). I'm sure there are more points to address within IT security, but this is a start. | This is something that has to be addressed in conjunction with the appropriate IT offices within the Center and the Agency. |
| Standards | Should we include some non-officially approved standards, such as some specifications developed by OGC or some de facto public standardized issues, such as web interface. Make a clear definition of work of standards WG and other WG, and their relationships, such as cost benefit, cost estimation. | The process is meant to include specifications from any source. The value of the specification is in how well it meets the needs of the user community, not in what its pedigree is. The relationship between the SPWG and other WG's will evolve over time, with the primary drivers being the "cross-pollination" of ideas by individuals who participate in more than one WG. Cost benefit and cost estimation do come into play when choosing among competing standards. The process of requiring independent implementations prior to final adoption of the standards will result in those specifications that are easier (and hopefully less costly) to implement becoming standards. Furthermore, the process will allow new specifications that might have different characteristics, including being more cost effective to employ, to become standards and to eventually displace older, less effective ones. |
| Standards | I think it might be useful to expand this a bit more. A number of different issues were raised at the Standards 2 session that raise the potential for additional clarification. Questions raised include which standards bodies, who decides to what level of participation? Something along the lines of "monitor relevant developments in national and international data systems standards organizations and where appropriate, resources permitting, participate . . ." might help clarify some of these issues. | The charter has been revised such that the list of responsibilities includes the statement, 'Participate in national and international data systems standards organizations.' |
| Standards 2 | If objective #1 on the draft charter includes the notion of stepping back to assess the process itself, it might be helpful to make this explicit somewhere in the charter. For example, adding an item to the responsibilities list along the lines of "Periodically (review?) evaluate the standards process as it pertains to meeting the SEEDS mission and where appropriate modify the process." The point is w/o clarifying where this responsibility lies, as well as the scope, this could be taken over by the SEEDS program office potentially. Arguably this should be part of the community consensus process encompassed by the standards process WG. | In the charter responsibilities, we've added: 'Periodically review and evaluate the standards process as it pertains to meeting the SEEDS mission and where appropriate, modify the process.' |
| Standards | There may be a need for a cross-reference to the "Life Cycle" efforts for archive standards. Substitute "Goals" for Objectives in the Standards Charter and use "Guidelines" to convey how existing standards are used in SEEDS. | 'Goals' has been substituted for 'Objectives' in the new revised Standards Charter. Tim Smith has reviewed the proposed edits to the charter and is comfortable with them. |
| Standards | 1. Explicitly include others in working groups (e.g., reuse) in the standards process, especially the approval or the "path to approval." 2. In the approval process, include the cost of adopting a standard. In other words, look at both technical and programmatic aspects before approval. Suggest programmatic review to be held after technical merit is determined and only when technical review is positive. | 1. We do understand that the standards process must have interactions with or input from the various working groups. The working groups are all tied together by the SEEDS office and the SEEDS office representative on the Standards Process Working Group is responsible for assuring input. 2. The cost of adopting a standard is important, but the standard process does not address it directly because quantifying such cost is very difficult and uncertain. Instead, we insist that standards be shown to work in our context. If standards have been used in an operational mode, we can have confidence that the costs are acceptable. |
| Standards | Should include an item like "Maintain awareness of relevant NASA standards activities and draw on those resources as necessary." The final class of standard is labeled as "ESE STD." Recommend changing to "SEEDS STD." It is not clear that SEEDS standards process applies to all of ESE, e.g., Applications. | 1) The charter has been revised such that the list of responsibilities includes the statement, 'Participate in national and international data systems standards organizations.' 2) We propose that SEEDS standards are ESE standards and apply across ESE. |
| Standards | 1. Change last sentence of first paragraph as follows, "Through management of the SEEDS Standards Process, the . . . relevant to data stewardship, the interoperation of ESE data systems . . ." to explicitly recognize that standards for data stewardship and preservation are also important. 2. Reference models and reference architectures, etc. also need to be explicitly recognized--the process can handle these, but it is not clear that things that are not quite standards in the strictest sense are included. | 1) Good point. The sentence will be changed to include data stewardship and preservation. 2) The process relies on RFCs that are not only standards but can also be a tech note. A tech note could be a reference model or a reference architecture. SEEDS does not presently propose to develop an Enterprise reference architecture but the standards working group can choose to take on that work after it has been formed. |
| Standards | Other working groups may very well wish to identify where a core standard is needed (e.g., life-cycle group may propose a standard for information needed to ensure long-term data preservation [e.g., the OAIS Reference model]). How do topic specific standards fold into the concept of the SPG holding the right to declare that something needs to be a core standard? | To address this issue the SPWG is composed of a variety of members from the SEEDS office, ESE mission projects, ESE data system awardees, ESE science data providers that should represent (or recognize) the needs of the ESE community. The SPWG will have to consider the proposer scope of a proposed standard on a case by case basis. The factors will include the proposed scope, the breadth of operational experience, level of interest, and other factors. |
| Standards | I have been trying to think about how this proposal would have impacted our Aura guidelines development. Our initial "strawman" document might be similar to the RFC. The Standards Process Group is similar to the DSWG and the Technical Working Group is similar to our Authors. With those assumptions, I have tried to walk through our process. The Standards Process Group set up the initial Technical Working Group with "minimal membership." Others were allowed to join. Each member was allowed to choose being either a "named author" (responses required for changes) or "silent author" (responses welcomed, but not required). The inital RFC was modified significantly by the Technical Working Group. Over time, major releases were approved by the Standards Process Group. We have had two major releases of the Guidelines and are currently working on our third. This will probably continue to evolve up to launch and may even evolve after launch if a major flaw is discovered. Hopefully, we are nearing the end of the modifications. Impacts I see from your process could be a slowdown of the process by requiring more documentation. I also see a slowdown of decisions due to the increased number of members and the length of time to reach consensus is probably longer. These are probably all okay because you need to have more structure and greater participation than we needed. I would suggest that you keep in mind the balance of "documentation and structure" with the "grassroots consensus effort." It is important to not put too many directives on the process, but to allow the members to have the ownership of the process and the results. Above all, make sure the members understand the benefits of adopting a standard. Also make sure there are members who are experts with using the standard on the Working Group. | It's interesting to see how closely the Aura guidelines 'community standard' process parallels the proposed SEEDS standards process Ð could indicate that this process may work well for ES data systems. RFC evolution: I would expect standards track RFCs to be somewhat more stable than the Aura initial strawman document. IETF area groups typically develop an idea through 'Internet Drafts' before publishing an RFC. Current Aura Guidelines probably correspond to an IETF Draft Standard Ð mature enough to code to, but may change to solve specific problems encountered in operational use. Our process should accommodate several iterations of an RFC. Need to flesh this part of the process out more. TWG membership: TWG containing a combination of appointed and self-selected members is a good idea. 'Named' and 'silent' authors: perhaps 'silent' authors correspond to 'public' in our process, interested parties whose comments are welcome.. Agree TWG should include experts on the proposed standard being evaluated. Possible slowdowns: It's appropriate for a 'core standards' process to be more rigorous than a 'community' one, since more people will be affected. However, the SEEDS Standards process may not require much more documentation, if Aura Guidelines are roughly comparable to an RFC. We have not yet defined what documentation is needed to demonstrate implementations and operational experience. It's hard to compare the number of Aura guidelines authors with the expected membership of a TWG, since neither number is defined. Agree TWGs should not be so large as to impede decision making |
| Chapter 1/LOS | This appears to refer to what in section 4.0 is termed the "Active Archive" as opposed to the Long term archive. This should be clarified. | The requirements and levels of service provided in Working Paper 5 'DSP Model - Requirements and LOS' constitute a general Data Service Provider reference model. Working Paper 6 'ESE Logical DSP Types' shows the mapping of Data Service Provider types described in the NewDISS study report with additional logical types, Applications Centers and Long Term Archive Centers, to the general model. The 'Active Archive' term used in Section 4 of the Recommendations Report corresponds to the 'Mission Data Center' in Working Paper 6. (Section 4 does not address the 'Backbone Data Center' role.) A note to that effect will be added to section 4 of the report. |
| First bullet refers to "robust" media without defining what that means. Need to define a minimum expected lifetime for media. Note that in many cases, the limiting factor could be the availability of drives to read the media, and not the lifetime of the media itself. | Because the limiting factor is indeed the availability of drives to read media, or the technical obsolescence of media and drives that make a migration to new technology cost effective well before the expected end of the media lifespan, there is no point in specifying a minimum media lifetime per se. The key is selection of archive media that will provide reliable, secure storage for the projected lifetime of the current archive system. Working Paper 5, Archive item 2.4.i will be revised, dropping the term 'robust', as follows: 'The data service provider shall use archive media that will provide secure, reliable storage of data and products for the planned life of the archive system.' |
| Third bullet states that there needs to be secure storage of "all standard or other science products". Is there an intentional exclusion of virtual products? | In a follow-up conversation, you described a 'virtual product' as a non-standard most likely higher level product produced on-demand for a user. There is no intent to exclude such products. The language of Working Paper 5 Archive item 2.4.c is 'standard or other science products' which would include 'virtual products' as you have defined them. |
| For "Range of Services" entry "Archive Media", it is stated as a minimum requirement that the archive media be vendor independent. For ECS, the current archive media is available only from Imation. While such a goal is laudable, it probably should not be a minimum requirement. | Dependence on a single vendor for archive media and/or an archive system has in the past given rise to painful experience and is a high risk state of affairs to be lamented if it temporarily can not be avoided. The Working Paper 5 Archive section item 2.5.i is being modified; the level 'archive media consistent with best commercial practice' is being dropped as too vague, and replaced with 'archive media is standard industry COTS'. That level is likely to be the minimum level used until practical alternatives are available from industry. |
| Chapter 1/LOS | The requirements here do not agree with the LOS "Production, Archive, and Distribution" requirements on page 36. In table 1.1, the requirement for Access is that there is "Access to a limited team of scientists or application users." Page 36 states that "All standard data products available to a science team member will be made available to general science users" and "All standard science data (Level 1b, Level 2, and Level 3) produced will be made available to any user who requests it without discrimination." | In Working Paper 5 Search and Order items 2.5.1 and Access and Distribution 2.6.1, the second and third levels of service will be modified to 'access to the science and applications community, with at least a minimal capability for public access', and 'access to a limited team of scientists or applications specialists, with at least a minimal capability for access to the science and applications community and the public'. A note will be added to both items: 'Public access may be limited in some cases by constraints levied by the original data source. Data service providers with a highly focused primary mission (such as a flight project instrument team) may meet the requirement for public access by teaming with other data service providers actively engaged in public access and services.' |
| The requirement under shipping for distribution in table 1.1 states that products must be shipped within one month of the request, whereas page 36 says that the average must be less than 5 working days. | The requirement under shipping for distribution in table 1.1 states that products must be shipped within one month of the request, whereas page 36 says that the average must be less than 5 working days. |
| The recommendation for (electronic) distribution states that the availability of a single product for access by user software will be "within 10 seconds". This recommendation cannot be met by a tape system. To load and thread a 9940B tape takes 18 seconds, and to position the tape to read the last file on the tape would take an additional 80 seconds. Is it truly the recommendation that all data be on-line rather than near-line? | There is no blanket recommendation that the highest level of service (10 second availability) be recommended for all data service providers. Designation of this or other level of service, such as the second level of service, a ten minute availability, as minimum or recommended for a future data service provider will depend on its particular mission responsibilities. It is anticipated that the cost of providing on-line access will continue to fall, and that user demand for on-line access will continue to increase, with the point of cost benefit balance shifting accordingly as time goes on. |
| Chapter 3/ Cost Model | The requirements here do not agree with the requirements on page 20, table 1.1, under heading "Access and Distribution". In table 1.1, the requirement for Access is that there is "Access to a limited team of scientists or application users." Page 36 states that "All standard data products available to a science team member will be made available to general science users" and "All standard science data (Level 1b, Level 2, and Level 3) produced will be made available to any user who requests it without discrimination." The requirement under shipping for distribution in table 1.1 states that products must be shipped within one month of the request, whereas page 36 says that the average must be less than 5 working days. For the first bullet, suggest changing "All raw data will be acquired will be calibrated" to "All raw data acquired will be calibrated". Requirements do not seem to allow a blackout period for QA. | In Working Paper 5 Search and Order items 2.5.1 and Access and Distribution 2.6.1, the second and third levels of service will be modified to 'access to the science and applications community, with at least a minimal capability for public access', and 'access to a limited team of scientists or applications specialists, with at least a minimal capability for access to the science and applications community and the public'. A note will be added to both items: 'Public access may be limited in some cases by constraints levied by the original data source. Data service providers with a highly focused primary mission (such as a flight project instrument team) may meet the requirement for public access by teaming with other data service providers actively engaged in public access and services. The Working Paper 5 requirements and levels of service do not include a specific requirement that all data acquired be calibrated, leaving the processing decision up to the data service provider and its particular mission. |
| Chapter 4/Life Cycle | In the second bullet, do you really mean "all" documentation, or just the most recent version of all documentation? | By stating that the missions should update the information, we assume that they will keep the latest version of the documentation. |
| Chapter 4/Life Cycle | For the fourth bullet on page 43, what does "state-of-the-art media" mean? Is it just media of a format currently supported by the active archive? For the next to the last bullet, documentation is to be provided in hardcopy form where possible. Why wouldn't it be possible? What will be done with the hard copy? Is it stored somewhere? Does it get transferred to the long term archive? The document is also to be delivered in a current softcopy format, and the sample format given is Microsoft Word. Two problems are that (1) there is not one Microsoft Word format, and (2) Microsoft Word is proprietary. Alternatives include PDF and OpenOffice. For the last bullet, what is the form of the training class? Is there documentation that is delivered as part of the training? | State of the art media as negotiated with the Active Archive... point is that the Active Archive cannot be expected to accepted any, outdated media. Change possible to supported... all organizations may not support hard copy. Documentation formats are just examples. Specifics on training courses are TBD... but we see them as an aid in maximizing the long term utility of the data. |
| Chapter 4/Life Cycle | Section states that the Active Archive is responsible for "processing", "data processing" and "data re-processing", however, these would appear to be the responsibility of "Science Product Generation" (Section 4.4.3), which is often the same organization as the Active Archive but is a logically distinct function. | The Active Archive supports the data processing functions. Will edit to be consistent. |
| Chapter 4/Life Cycle | In the third bullet on the page "Communicate archive requirements of the individual instrument teams", to whom is this information to be communicated? LTA? Is this the same as the fourth to last bullet? In the seventh bullet on the page, how does one "design the files" to be "easy and efficient to archive" and why is it that the active archive, rather than the "Science Product Generation" that is responsible for the design? The last bullet says to allow for production on demand systems. The second bullet on the page says "Plan for archiving all instrument data". Are production-on-demand products archived? Does the LTA provide production on demand, or is that capability lost once the active archive is no longer involved with the mission? | The "communicate" bullets are redundant and ambiguous... will clean them up. "Design the files" should be done by the Product Generation in consultation with the Active Archive. Archiving "production on demand" products or doing production on demand at an LTA are possible but decided case by case. |
| Chapter 4/Life Cycle | In the third bullet, to whom is the LTA supposed to communicate the requirements for LTA? | To the data producers... will clarify. |
| General | Foundation comments: Many of the SEEDS contributors are Federation members. The Federation seems to be best-positioned to be involved in metrics. It has a mechanism for collecting metrics, including "nuggets" and raw metrics. The Foundation is keenly interested in reporting and performance data as it looks to highlight key member successes in its fundraising and general Federation marketing efforts. We would like to see NASA extend its metrics focus to include, when appropriate, socio-economic impact of its funded data projects. | Thanks for the recommendation about Federation's involvement with metrics. We have been coordinating the metrics study with the Federation since almost the beginning of the study. The SEEDS Metrics Planning and Reporting study team has had active participation by the members of the Federation (Howard Burrows, Frank Lindsay, Hank Wolf, Don Collins and Steve Kempler - the last two being Federation partners as type 1 ESIPs). Bob Chen, the Manager of the Socio-Economic Data and Applications Center (SEDAC) also participated briefly in the study team discussions. We have also had discussions of the federation metrics, including "nuggets" both within the study team and in the community workshops' breakout sessions. We plan on working with the Federation's metrics group as we convene the SEEDS Metrics Working Group. Certainly, the socio-economic impacts are valuable to address and report on as we proceed with this working group. All the available tools to facilitate collecting and reporting on metrics will be looked at, including the Federations' tools. |
| Standards | There is an overemphasis on format standards, rather than interface standards. The Federation's GIS Services Cluster has enabled numerous sites to install WMS servers. We have successfully demonstrated WMS usage. Also there is under emphasis on web service standards that are widely used in business. | We recognize that there are many other standards in addition to data formats. System-system interoperability requires defining the interface standards by Data Service Providers. We hope to define such interface standards with community participation and adopting approaches from other organizations such as ESIP, OGC and W3C. The adoption process will facilitate easy adoption of standards mentioned above. |
| Standards | First of all, SEEDS recommendations can help us building an interoperable Digital NGP. For example, it recommends several file formats, software used to write metadata, what format should be for metadata, and where to report metadata. Secondly, while SEEDS highly emphasizes the interoperability, I feel that it is more on the data format side, e.g., HDF, GeoTIFF, etc. What we like to see is the system-system interoperability, which is that one system can serve as a proxy of a user to retrieve the data from another system transparently to the user. To do this, we need to know how to access data at a system level. From the SEEDS' point of view, each DSP (data service provider) should provide a description on the data access at the system level. This is critical in linking all the DSPs together to form a nation-wide network. This could go to either Level of Service or Data and Information Standards. | We recognize that there are many other standards in addition to data formats. System-system interoperability requires defining the interface standards by Data Service Providers. We hope to define such interface standards with community participation and adopting approaches from other organizations such as ESIP, OGC and W3C. The adoption process will facilitate easy adoption of standards mentioned above. |
| Introduction/goals | The stated SEEDS goals are very similar to those for the ESIP Federation: 1) maximize utility of products, 2) leverage the community, and 3) improve services. In spite of this similarity, it almost seems that the recommendations and even the Study Team topics themselves eliminate consideration of many of the lessons learned in the Federation. For example, the possible benefit of bringing the science community closer to the decisions concerning the data doesn't seem to fit in as a recommendation. | To address your specific example, the MPAR group has extensively discussed, and continues to discuss, the appropriate role for the science community within the context of a SEEDS Office. The important difference between the two groups is that the Federation is self-formed from organizations receiving money from NASA in order to collect, process, and disseminate data, while a SEEDS Office would be the government entity, potentially responsible for giving the money to those organizations, as well as making strategic decisions regarding the future of ESE data systems. The cleanest mechanism by which to involve both the science community and the funded data providers may be via the working groups established to run the unifying framework. Those working groups each tackle issues of importance across the board, and those working groups encourage participation by all parties interested in that topic. SEEDS is not solely an advocate for the science or data community. As a government entity, NASA, and by extension all of its projects, has a responsibility to spend taxpayer money wisely. It is not wise use of taxpayer money, nor, in some cases is it legal, to allow the entire community to make all the decisions. While we feel that, at least initially, the use of the working groups strikes the appropriate balance, we will try to get this point across when discussing implementation. (The recommendations document is designed solely to present the results of study teams and not to present an office implementation with all associated structures, interactions, and benefits) . It is also important to realize that the ESE has learned many lessons from the development of the EOSDIS. One of the major lessons is that the Enterprise should not over promise what it can do. One of the primary ways it ended up overpromising was to go directly to the science community and ask what it wanted. We have taken that lesson to heart, and are careful not to overpromise, an |
| Introduction | How do you even comment given the seven study teams as summarized in the table on page 9. Nowhere do the teams address who is in the stakeholder community, let alone how to engage them. Yet involving stakeholders is one of the three stated goals. | To address your specific example, the MPAR group has extensively discussed, and continues to discuss, the appropriate role for the science community within the context of a SEEDS Office. The important difference between the two groups is that the Federation is self-formed from organizations receiving money from NASA in order to collect, process, and disseminate data, while a SEEDS Office would be the government entity, potentially responsible for giving the money to those organizations, as well as making strategic decisions regarding the future of ESE data systems. The cleanest mechanism by which to involve both the science community and the funded data providers may be via the working groups established to run the unifying framework. Those working groups each tackle issues of importance across the board, and those working groups encourage participation by all parties interested in that topic. SEEDS is not solely an advocate for the science or data community. As a government entity, NASA, and by extension all of its projects, has a responsibility to spend taxpayer money wisely. It is not wise use of taxpayer money, nor, in some cases is it legal, to allow the entire community to make all the decisions. While we feel that, at least initially, the use of the working groups strikes the appropriate balance, we will try to get this point across when discussing implementation. (The recommendations document is designed solely to present the results of study teams and not to present an office implementation with all associated structures, interactions, and benefits) . It is also important to realize that the ESE has learned many lessons from the development of the EOSDIS. One of the major lessons is that the Enterprise should not over promise what it can do. One of the primary ways it ended up overpromising was to go directly to the science community and ask what it wanted. We have taken that lesson to heart, and are careful not to overpromise, an |
| LOS | I think we said every early that to do "levels of service" independent of value was strange. Why bother talking about "costs" without reference to benefits? | The Data Service Provider reference model, which includes the requirements and levels of service, is intended to provide a general framework for describing the range of ESE data service providers and to underpin the Cost Estimation Tool (CET). When a particular data service provider is being planned, the appropriate requirements and levels of service can be drawn from the general model as appropriate for its mission, and the 'benefits case' can be made by the planners (at first the program officials who see the need for the provider to meet program goals, who must win budget authority for it, and who develop a procurement action for it, and subsequently the proposers who respond to the procurement action) with reference to its specific mission, while the CET can be used to produce a life cycle cost for it. |
| Introduction | Many decisions for the study teams such as choice of standards or allocation of resources to interface development, new technology, or long-term archiving depend on cost and value. Where do they even discuss "value to what?" Is it to science, to intriguing congress, to business and the economy, to saving life on Earth . . .? | Within the ESE, obtaining data in order to do research to study the five fundamental issues of climate change, is the mission. Everything we do, without exception, must add value to the data product so that the highest quality research can be done, the highest quality product can be used in a given application, and the highest quality product can be used to train the next generation of earth scientists. Outside of the Enterprise, the value of our data products is determined via the applications and decision support systems developed. We do not, and cannot, quantify in dollar terms the value of knowledge. We can indirectly quantify the amount of business a particular company does using NASA imagery, but this entire discussion is embedded in the discussion of outcome metrics. Those are very hard to create, and the MPAR group has as one if its main tasks, to develop outcome metrics which attempt to answer the question in the traditional, commercial sector sense. This discussion has a way of coming up when one weighs a public good against a profitable venture. The two should not, in my opinion, ever be compared on the same scale, but often are. |
| Metrics | The Metrics team studied only those metrics needed for accountability to NASA managers. The choice of metrics that would be required to compare products and services across DSPs in order to build teams and value chains doesn't seem to be there at all. There doesn't seem to be a process for using the metrics to establish the perceived or real value or 'utility' of the products and services to specific communities. | We are in the early phase of identifying and defining metrics for SEEDS. An MPAR WG charter has been drafted and a community working group is expected to be formed soon to deal with metrics. We encourage participation in the working group by persons experienced in building such value chains and defining the appropriate metrics. |
| General | Dave Observations: It is critical that the private sector (data and information services) will need quick access to data to service its clients. In addition, it is critical in order to address NASA ESE's vision out to 2025 that cross-agency participation in earth science-related activities and events be made relatively seamless. To do this means enabling SEEDS to address data issues outside of the Enterprise. This is a challenging task. It will be cost-prohibitive for private sector organizations to develop disparate solutions to processing various agency data and information. Partnerships are needed. | Agreed. When the SEEDS Office is established, one of its primary responsibilities will be to work with other agencies to develop and maintain data related policies which are consistent with the current Administration's goal of commercializing as much remote sensing capability as is reasonable given national security considerations. In addition, NASA is deeply involved with the CCSPO, which is the principle mechanism for such federal partnerships for the foreseeable future. |
| Introduction | Relationships Among Government, University, Commercial Data Providers and Stakeholders. Cooperative Agreements and relationships among all involved should be established. It is critical that any organization in the future that receives funding from NASA or any other agency as it relates to earth science should adhere to the principals and practices of responsible data preparation, storage, archive and distribution. | Agreed. The appropriate funding mechanism for each organization may not be CAs, but we agree with your comment. |
| Introduction | Future missions must plan for success and plan for quick transition to operational use of the data. Plans must be made therefore to expedite the transition from research to operations. | This is something that the Agency has determined it must do in partnership with NOAA. NASA is 'getting out of the business' of operations and is returning to a more research-oriented organization. NOAA is taking the lead on the operations side, and the transitions are still being worked out. |
| General | SEEDS should accelerate the ability to show the world that earth science can address global issues quickly. | In a way, this is the job of the Applications Division at Code Y. They are working very hard at making the case that Earth science data feeds into decision support systems which are critical to resolving issues of national importance. In fact, the REASoN has just awarded some of those systems. As far as global issues are concerned, it is the science community with their peer review process that shows the world how NASA data contributes to the understanding of global climate issues. The only realistic role of a SEEDS Office in this context is the development of some sort of citation metric that can be collected and reported. Our MPAR team is working on that. |
| General | What is critical to understand in this era of sustainability is: To what extent do future sensors NEED to produce the huge amounts of data scientists think are necessary to better understand the earth system. How much data is enough? (Dave's 40% Rule). Future missions must understand more clearly the implications of gathering so much data. What drives the costs of ground processing systems, distribution systems, archiving and access systems? Even in a distributed environment... can the distributors keep up? | Agree. This is a decision that will be driven as much by long term archive policies being developed between NASA and NOAA as by anything else. But the most important hoop to jump through right now is the one created when one separates real-time needs from research needs. In the real-time community, less and focused is better. In the research community, more and broader is better. Those two communities have far different views of what is important and what should be produced and saved. This is a policy question that SEEDS should help address once the office is established. |
| LOS/Working Paper Five | I recommend deleting this requirement altogether. Rationale: Hardware cost is declining so rapidly that it is becoming a negligible portion of the total budget of a science investigation - particularly in terms of a budget that includes both hardware and people. As identified in the first pdf document attached, we are at a point where the cost of extending the mission because of hardware under-capacity already makes 10X an optimal production capacity for simple models of production. It is clear from the calculations that the minimum is already becoming quite flat and insensitive to the minimum value. The second attachment extends these calculations somewhat and considers additional conditions - with the same conclusion. I've done even more complex calculations - with the CERES schedule - and have essentially the same conclusion. | The requirement, Processing item 2.2.c in Working Paper 5 'DSP Model - Requirements and LOS' will be dropped. Reprocessing will be handled by the CET on a product type by product type basis, with the user providing a nominal reprocessing interval for each product type as desired. |
| The important lesson from this work is that we need to get the science teams to give us better schedules - but to treat these entities as stochastic in nature and ensure that the teams have the ability to base their need for contingency on a more solid base of experience than we have had from them in the past. |
|
| LOS/Working Paper Five | I'm not quite sure what to do with this LoS statement. I can identify eight production paradigms in EOS, ranging from "pretzel-making" for Landsat, "Motophoto" for ASTER, various levels of batch processing ("Simple Machine Shop" for MOPITT, "Moderate Load Machine Shop" for MISR, "Transfer Line" for MODIS, and so on). Landsat is simple and could be done with a negotiated schedule. ASTER is "production on demand" - for which production capacity can probably only be estimated on a statistical basis. CERES, at the extreme end, is "priority interrupt scheduling," which requires a stochastic approach that might try to base capacity on stochastic optimization. The latter approach is known to be NP-hard in some cases - and is likely to involve serious Monte Carlo or simulated annealing algorithms. To make the situation even more complex, it is also clear that the production and validation scheduling depends on the intent of the investigation. If a data set is intended for a community that have "operational" needs for near real-time data, and if the data use is insensitive to calibration, then it is possible to provide useful data in near real-time (3 to 6 hours). On the other hand, if the data set is intended for climate data use, it may not be usable until after a six to eight month period Ð and may have to have a multi-year (two to four years to be a bit more precise) validation period just to make sure the data set has homogeneous uncertainties. | The general requirements / levels of service statements, Processing items 2.2.a and 2.2.b in Working Paper 5, taken together generally embrace the scenarios described in the comment, and provide a framework within which more specific requirements could be written for a procurement action for an actual data service provider. The Comparables Database and Cost Estimation tool work with annual aggregates of workload and some tuning based on the characterization of that workload as a mixture of operational and non-operational processing, without getting into levels of detail that might offer more precision in the result, but only if correspondingly precise information could be collected from ESE data activities for the Comparables Database. Information at that level of detail has not been obtained, though it could be sought if results achieved with a more general level of information prove inadequate. |
| LOS/Working Paper Five | These two sections are confined to the current paradigms, in which users search for granules, order them, and have the files delivered by one means or another. There are much more interesting paradigms for user data access that become particularly relevant under conditions of grid computing. These include "interactive, content-based search", "search and retrieval based on (irregularly shaped or anomaly based) object catalogs" - extended annotations, and autonomous, intelligent agent data search, summarization, and retrieval. I believe that the falling price of CPU's, storage, and networks will make these approaches dominant in the not very distant future - and will vastly reduce the cost of data access and use. Thus, both of these sections strike me as forcing the assumptions of the early 1990's on the vastly different IT environment of 2003+. This will be particularly true if grid computing lives up to its full potential (see the current article in Scientific American for an example of the vision). | The requirements and levels of service in Working Paper 5, Search and Order, Accessing and Distribution, and Processing allow for content based search, system-system search and access, allowing user 'intelligent agent' software to run within the data activity's system and interact with the user's system, and allowing user-provided software to 'data mine' and generate products and deliver them. The reference to spatial search at the highest level of service will be revised to include '... and space (by named spatial object from a catalog as well as by coordinates)... '. The intent is to be open to possibilities in the future, e.g. grid computing, and the requirements and levels of service will be maintained and evolved as technological possibilities become realized and cost effective. |
| General | As an additional note, the third attachment is a tutorial on computer performance prediction that includes a fair amount of research - both in the literature and from personal experience. What may be of interest from a cost modeling standpoint is the method of extrapolating from an estimate of production capacity early in a science team's life to one that includes operational experience. Table 6 on page 34 represents a summary table that allows a science team to provide a multiplier of their estimates at various stages in the project's life and their risk aversion - providing a "rational" basis for the estimate based on data from the CERES experience. This may be a limited experience base (I think I can get data from other teams), but I think it's the first time any of us have had an empirical basis for doing such estimates. The data also do not support the notion that science teams have an exponentially increasing appetite for processing power - although I think there's some experience with such an appetite on the part of data system developers. | As development of the Comparables Database and the Cost Estimation tool proceeds, the LOS / CE team will continue to review this work and its future progress. The cost estimation by analogy approach is constrained by the completeness, consistency and level of detail of information available from ESE data activities, which will limit the level of detail at which the cost estimation can operate. The general approach will be to obtain the best results possible with the level of detail that now exists in the Comparables Database, i.e. extending it to include more data activities and refining the estimation approach in each functional area, and then to analyze and identify the weak points in the Cost Estimation Tool and/or Comparables Database and work to improve them. That could include going back to the ESE data activities for more detailed information in key areas as well as more work on the cost estimation approach itself. |
| Not specified | The issue of 'one size does not fit all' appears implicitly and explicitly in a number of places in the draft SEEDS documentation. However, there is no description of how the appropriate fit will be determined. Is this something that will be determined by the working groups, via an RFP, through contract/budget negotiations between NASA and a potential contractor, or through some other means? | The appropriate fit will be determined by the community via the forum of the particular working group. Implementation of that 'fit' might be done via solicitation of some sort. There are two phases that will happen within the office. One phase will be done largely by the working groups and conferences, where issues are raised, solutions are presented, debated, and researched, and candidates are sent out for solicitation. Metrics are collected to see how well that solution did, and the cycle continues. The other phase would be to work from the Enterprise level to identify long-term goals and direction, and then communicate that to the working groups. |
| Not specified | The draft recommendations outline the creation of a series of working groups addressing various aspect of data system evolution. Arguably, there is overlap with some of the existing ESIP Federation Committees. Has or will there be any discussion of potential reconciling of this issue? As an example, the REASoN CAN required an up front commitment to participating in SEEDS working groups as well as application to the ESIP Federation. These aren't exactly the same, but are potentially redundant. | There is a fundamental difference between the Federation and SEEDS. The Federation is comprised of government-funded data service providers who have joined to form an entity which will, hopefully, eventually, acquire broader funding. SEEDS is a federal government effort which will provide some measure of direction to, and may be responsible for providing that funding to Federation members. What the Federation members do in their working groups should be more focussed than what SEEDS wants to do. Federation working groups ought to be focussed on how to grow the Federation and make the best use of Federation systems and services. SEEDS has to concern itself with *all* ESE-funded data systems (not just the ones within the Federation), *and* with the direction the Enterprise and its associated Research and Applications Divisions are heading. Having said that, there is expertise in the Federation which is of value to SEEDS, and SEEDS has indeed 'mined' that expertise to a great extent in its study teams. So, while there is overlap on the surface, there should not necessarily be overlap of working groups between the two entities to the extent that one set of working groups becomes redundant. The Federation is trying to become self-sustaining; SEEDS has appropriated funds. There is the potential for a great deal of sustained partnership, but not for elimination of one working group or another. |
