Information Management Tools

Last updated: 20 Dec 2022

Overview

Information management tools range from generic data collection tools to IOM-specific solutions, including the Displacement Tracking Matrix (DTM) data dictionary, DTM standards, database, central data warehouse; they also include data analysis tools such as Kobo toolbox, Excel, Tableau, SAS and Stata; and data visualization tools such as Excel, SAS and Geographic Information Systems (GIS) tools, online tools and servers. These tools are described in more details in this section.

Description

Data Collection Tools

To collect data, you may choose to use a Paper form in the field, to be followed by data entry through an interface (tablet / computer) or may opt to collect data directly on a mobile device, using applications such as KoboCollect or Open Data Kit (ODK). These applications have been optimized for use in remote locations, and do not require internet access at the time of data entry – the information can be intermittently saved on the mobile device and transferred to the server at a later stage, once a reliable connection is obtained. IOM recommends the use of Kobo due to its user-friendliness and enhanced capabilities such as enabling real-time control (as soon as surveys have been submitted), including data cleaning, geo-referencing, and basic analysis.

In some settings, it is recommended to undertake field data collection on paper even if a mobile data collection platform is later used for data entry in the office. Advantages and disadvantages for using paper forms or mobile data collection in the field:

Paper Form	KoboCollect
(+) It always works	(-) Technological hiccups are possible
(+) Easy to use	(-) Some familiarity with technical device required
(+) More flexibility in the order of asking the different questions	(-) Swiping back and forth can be cumbersome
(+) It's possible to take notes	(-) A paper is still needed for notes
(+) It does not introduce additional tension in the interview	(-) In some contexts, it might create suspicion in the interviewee to see an electronic device (especially when interviewing migrants or IDPs directly)
(-) Errors can be introduced at data entry stage	(+) Data entry is immediate, avoiding typos or errors when interpreting handwriting
(-) Many paper, requires storage space	(+) Paperless, easy to store electronically
(-) Delays caused through the data entry stage	(+) Real time control and basic analysis are possible directly after forms have been submitted
(-) Paper forms, if not stored securely, can lead to breaches in confidentiality	(+) Confidentiality of data can be strengthened provided the server is secure

You can develop a data collection form using samples from other country offices or using the CDW. Samples of each type of form are found in the resource section. Using the data dictionary is essential to make global analysis of data possible.

Considerations for working in an environment with little internet or electricity: There are certain smartphone brands that have long battery life. There is also the option of buying good solar batteries though it can be difficult to find reliable and durable ones. Also consider adding Locking Apps onto the phones like ‘Smart App Lock', this ensures enumerators cannot use the phone for anything other than the intended purpose.

Central Data Dictionary

Whilst implementation of information management activities varies somewhat from country to country, and local adaptation and contextualization are essential for the production of operationally relevant data, often very similar data is collected in the different interventions. The use of the DTM data dictionary, a repository of questions asked in previous implementations and indicators that were established as essential when collecting information, enables coding of question-and-answer options to facilitate comparison of data from different country offices. The data dictionary has capacity to have translations to any language and can be easily imported into KoBo. As of now, the dictionary can be found in English, Spanish, French and Arabic. The DTM data dictionary is a living web application. Access to the Data Dictionary may be requested through the http://dtmsupport/datadictionary or by contacting [email protected]. Access is granted on a case-by-case basis.

The main purpose of this platform is to accelerate data collection and analysis across all DTM operations worldwide. The integrated combination of indicator repository and the form builder/validation app has the potential to contribute to a more systematic data collection, more standardized data management & governance, thus improving the pace and efficiency of DTM operations. It will ensure that indicators are consistently collected with matching data types. This will enable swifter data cleaning, more automated data cleaning with the deployment of pre-designed scripts, easier data management, higher quality data and an environment for more automated analysis.

The objective of the Central Data Dictionary is to ensure consistency and minimum quality standards between DTM's various humanitarian, peacebuilding and development data collection initiatives. This will allow for DTM to maintain data consistency across multiple countries and regions and to have defined, coordinated, useful and responsible recommended standards that enhances DTM operations at the country level, regional and global level, consistent and with respect to data governance, security, protection and advanced ethical norms and practices.

DTM Standards

The DTM Standards specify the minimum requirements and recommended guidelines to support all DTM staff to successfully implement DTM programmes. The Standards apply worldwide across DTM activities and serve as a reference tool for DTM staff, IOM partners and other stakeholders and users, particularly in the use of DTM data.

Through harmonization of the various DTM guidance documents produced across operations and over time, the Standards make it easier to identify good practices as well as existing gaps. As they become available, the Standards will be posted to a centralized DTM Standards SharePoint space, to help all users to quickly locate and retrieve relevant requirements and guidelines.

The Standards are minimum requirements that recognize the importance of consistent safeguards, systematic monitoring and evaluation as well as comparisons of DTM exercises and data across time and context. Developed collectively by DTM staff from country, regional and global teams, the Standards also recognize the importance of flexibility and adaptability in DTM’s response to context-specific needs. Changing conditions, innovation and emerging practices will inform revisions to our Standards.

The DTM Standards cover the following topics:

DTM Standard Group	Description
AI, data science and ethics	These Standards provide recommendations for the full lifecycle of AI and data science projects in DTM, from background information and preparatory work to post-mortem assessments. This Standard also compiles a wide range of resources for further reading into specific issues, in addition to providing clear, IOM-specific guidance related to ethical, technical and practical concerns.
Data analysis planning	These Standards define the process by which DTM Officers, together with partners, identify and prepare (research) objectives, and detailed plans that specify roles and responsibilities, outline and list the type and levels of analysis to be performed from both primary and secondary sources of data, the data analysis (and visualization) production tools, process and formats to be used, as well as the limitations (and risks) of the data and sources.
Data archiving, recovery and destruction	These Standards define data archiving as the process of securely storing data not actively in use in DTM work. It defines data recovery as the process of retrieving information from archives for (return to) active use. It defines data destruction as the process of removing information from media (the material in which information is stored) in a way that it can no longer be retrievable or readable. This Standard describes the requirements and guidelines covering the classifications for the archiving, recovery and destruction of IOM/DTM data; how to securely archive, recover and/or destroy DTM data; the approval processes required; tracking DTM data archiving, recovery and destruction; and includes essential data archiving, recovery and destruction resources.
Data design and collection	These Standards provide requirements and guidelines on how to design and collect DTM data with the goal of producing high-quality data that is fit for use in both Humanitarian and Development settings as well as other DTM activities. This Standard defines the different steps involved in data collection from identifying the need for data up to data collection and validation. This Standard also provides guidelines on how to integrate data validation in the design process.
Data protection and privacy	These Standards provide a checklist of key steps to secure the right to privacy of data subjects. IOM defines data protection as the systematic application of the institutional, technical and physical safeguards that preserve the right to privacy with respect to the collection, storage, use and disclosure of personal data. This Standard is informed by IOM Data Protection Principles (IOM IN/138), the DTM and Partners Toolkit and other key protection related resources. This Standard sets out actions and processes to adopt at each stage of the information management cycle.
Data sharing	These Standards describe the requirements and guidelines covering all forms of data sharing including the process and approvals needed for sharing DTM data according to the IOM data classification and the Data Governance Policy (IOM IN/253).
Data visualization	These Standards provide guidance on the production of DTM data visualization outputs at the global, regional and mission level to ensure consistency and standardization across all DTM outputs. This Standard provides specific parameters to consider when producing visualizations; link to existing visualization sources (e.g. DTM Reporting Standards, DTM Data Visualization Design Guidance); identify gaps and define new standards, if and as needed; and outline the process needed for clearance/approval for external dissemination. This Standard will cover products that are narrative in focus (visuals for reports), maps or any other visual output, including static and dynamic outputs.
External relations and partnerships	These Standards provide guidance which includes clear definitions and procedures for engaging, initiating, and maintaining relationships. Also included in this Standard are typical classifications and types of partnerships — boundaries for engagement — and protocols for entering partnerships, coordinating thoroughly with IOM's Departments of External Relations, Legal Affairs, and other relevant focal points. Furthermore, this Standard draws on and expands, in parts, on already established literature within IOM.
Geospatial information	These Standards draw from the IOM/DTM Geospatial Systems Resource Guide and provides operational guidance in addition to providing definitions and standards on spatial data infrastructure, cartographic design, and visualizations related to geospatial data.
Internal data consolidation	Within these Standards, internal data consolidation is defined as the process of centralizing datasets produced by each DTM mission at the regional and global level. This document outlines the steps for sharing and validating a final dataset through different pipelines to ensure DTM data consistency.
Monitoring and evaluation (M&E)	Within these Standards, per IOM's Monitoring and Evaluation Strategy, monitoring is defined as the continuous function that uses the systematic collection of data on specified indicators to provide management and the main partners of DTM with an indication of the extent of progress and achievement of objectives and progress in the use of allocated funds. Evaluation is defined as the systematic and objective assessment of ongoing or completed DTM activities, their design, implementation and results. This Standard outlines existing M&E activities with good practices, presenting available sources and identifying the gaps in order to support the enhancement of DTM programme and data quality.
Project management	These Standards draw a foundation from the IOM Project Handbook, promoting it as the reference point and key institutional standard for project management. This Standard also identifies key modules in the IOM Project Handbook of particular use for DTM operations and provides context specific examples such as enumerator modality, project development checklist, and cooperation agreements.
Reporting and publication	For the purposes of these Standards, publishing constitutes the dissemination of DTM reports and data through official IOM/DTM web-platforms, recognized unaffiliated platforms (e.g. ReliefWeb or HDX), academic journals, social media or other means of public dissemination. The primary focus of this standard will be the publication of reports. This Standard covers: quality assurance protocols (including the review process), risk assessments for publications (data and political sensitivity), and DTM brand harmonization in reports.
Sampling	These Standards describe the definitions, requirements and guidelines covering sampling in DTM operations. Sampling methods, including designing a sample and sample frame, and implementation strategies are provided.
Training and internal capacity building	These Standards define training as the organized process by which people learn knowledge and/or skill for a definite objective. Within DTM programmes, training refers to the teaching and learning activities planned and organized with the goal of (applying) knowledge, skills, abilities, and attitudes needed to advocate for, plan, implement, report on DTM activities.  This Standard outlines the steps and resources to be used when developing, implementing, and monitoring DTM training activities for different audience groups.

Database

Database as a Data Storage media

With the different interventions that country offices have, there is much data that they collect. This data needs to be stored centrally and securely for reference and reporting. Using a database enables storing all the information centrally whilst using proper security configurations, enabling data to be accessed by more than one person without dependency on colleagues. Data can be stored in databases such as:

Microsoft SQL Server Database
MS Access and MS Excel (free and comes with the Microsoft Office package)
Comma Separated Values (CSV) files
MySQL servers

The common data storage for survey data is Excel and MS SQL. With the use of the data dictionary, it is possible to store data in one database server and hence making it easily accessible. The KoBo database servers are being used as a temporary data collection and storage before exporting the data to SQL serversor Excel for manipulation.

The recommended database to use is Microsoft SQL Server database. This is because it is easy to use and is widely supported. Users can easily connect to the SQL Server database using Microsoft Access, Microsoft Excel and other tools that are used for analysis. MS SQL Server database can store large amounts of data without posing any risk of losing data.

Central Data Warehouse

The nature of IOM emergency operations and absence of recommended corporate solutions in the past has led to disparate system developments at the field level. While there has been an overall information gathering framework under DTM, the way of implementation and tools vary greatly (MS Access, Excel, SQL database, ODK etc.). These scattered and different information systems located in various IOM field offices pose a challenge when the organization requires to have consolidated information bringing together data from different contexts.

In this respect, a central data repository has been identified as a necessity, to have the ability to provide reliable, consolidated, integrated and up-to-date data at a departmental and organizational level in order to provide an accurate consolidated regional and global view of human mobility data gathered in emergency operations. Such a central information repository incorporating data from across field implementations enables the use of historical data for trend analysis which can support better planning for future emergencies. The data stored into the CDW will also directly feed into various presentation systems, including GIS portal, dashboards, SAS, analysis and reporting services and other data users. For more information on Data Warehousing Concepts, see the References and Tools section.

The strategy for building up the CDW is as follows:

1. Field/country offices collect data and store in different formats (SQL Server databases; MS Access databases; Excel/Google spreadsheets; Documents, PDF/Word; Photos; etc.)

2. Country offices send data to the staging servers through APIs. A distinction is made between legacy data (and data which does not follow DTM data dictionary) and standard data (which does follow the data dictionary). Almost all country offices have internet connection and networks are configured to be within the Virtual Private Network (VPN) of IOM. For data security, the field offices will use VPN to post data to the central data warehouse.

Legacy Data is stored as raw files. Data in Excel/MS Access/CSV can be transformed and stored in each country office's respective database within the repository. Legacy data can be mapped to the standardized data dictionary.
Standard Data is stored in each respective ODK/Kobo server in field offices. Data is stored in the central data warehouse at the HQ for those who use the web interface for data collection.

3. Data integration and normalization is done in the central data warehouse.

4. Processed data is stored in the data repository and exposed to users.

5. Users access data through ready-made queries and use data services through OLAP and queries. Data sharing mechanism through web UI is implemented and made easy for data access, extraction and sharing. Each data user will define the nature and structure of how they need to view the data.

Data Analysis Tools

There is a variety of data analysis tools available. The tool recommendation depends on the level of complexity of analysis, the periodicity, and the size of the dataset to be analyzed.

Kobo Toolbox	Only recommended for control during data collection but not for outputs for reports. Basic analysis can be provided in real-time.
Excel	Most used tool in DTM implementations. It provides basic and advanced analysis capabilities on ad-hoc basis and on various datasets. Additionally, it is also a visualization tool. More advanced functions provide for automatic templates.
Tableau	Enables the creation of interactive dashboards with descriptive analysis that can be externally published on the web.
SAS	Features descriptive analytics capabilities: good for production, as templates can be loaded directly. Larger datasets can be processed. It is also a visualization tool to produce interactive dashboards. + Advantages of the SAS analytics platform include user-friendly interface and predictive analytics capabilities. - Currently, it is only for internal use. The output needs to be cropped using snipping tool for external use. - Requires paid licenses. Check with DTM Support if any are available.
Stata	This software allows for more advanced analysis, such as regressions + Easy to run commands in tandem, which provides for quick analysis on large data sets. Very poor on visualizations, so it still needs to be combined with Excel, SAS or another tool. - Requires some practice and as the software is not free/not part of standard IOM IT package, requires additional financial investment.

Data Visualization Tools

Since the recent developments of the DTM globally, the need to incorporate core standards in DTM products across country offices was raised. As such, global initiatives to standardize processes, approaches, and methodologies of the DTM were introduced. This includes certain tools that facilitate core DTM activities (form development, data entry, connecting to platforms for data visualization and analysis through the SAS Visual Analytics Dashboards or Esri Incident Management Platforms).

The current pilots of DTM data visualizations and key tools available such as Visual Analytics Dashboards, and Incident Management Platforms where DTM information products can be published on a regular basis show the functionalities of data visualization (from creating questionnaires to collecting data and connecting to visualization platforms), DTM App (application for collecting and reporting data) and SAS Visual Analytics Dashboard (interactive platform to visualize and analyze data). These key tools of DTM data visualization include:

Excel	It is the standard software used by IOM DTM operations that provides flexibility for producing basic graphs, such as bar chart, pie chart, line chart, crosstabs table, etc. There is possibility to automate the production of this graphs, but no possibility to interact with the objects.
SAS	IOM has piloted the use of SAS Visual Analytics and Visual Statistics platforms. The main feature in terms of data visualization of these two is the possibility to produce interactive objects (graphs and maps). The current arrangement of IOM with SAS provides the possibility to use a cloud server for the reports, meaning that to access the reports it is mandatory to use credentials. Thus, it is only for selected partners and internal use.
GIS	DTM maintains an internal GIS knowledge and resource sharing platform accessible to all GIS users worldwide. Powered by Microsoft SharePoint and ArcGIS enterprise, the platform hosts a catalog of GIS referential data, internal guidance, useful resources and links, external capacity building initiatives, and a community of practice for users.

Geographic Information Systems (GIS)

Geographic Information Systems’ (GIS) tools have a broad range of applications for IOM and DTM in particular, including improving operational efficiency through more informed programmatic planning and implementation using geospatial analytics. This is most evident in humanitarian responses within IOM in which accurate, reliable, and up-to-date data on displacement dynamics have lifesaving consequences. For external partners and audiences, GIS online visualizations often help display key results and indicators on easy-to-read map products.

The Geospatial Analytics team within the DTM unit provides the following services to IOM’s DTM operations worldwide:

Assist global IOM operations by contributing to the design, development, and maintenance of relevant geospatial data infrastructure and geodatabases to ensure the effective use of geographic information systems.
Ensure that common geospatial data standards and structures are applied, and data are managed properly throughout the data management cycle, including through support in the data collection processes, analysis and overall data management activities.
Coordinate, design and maintain the Central Data Warehouse (CDW) to ensure smooth data flow to geoportals, databases, applications, and other GIS solutions. The CDW is a core component of global DTM systems and infrastructure that supports quality, harmonization and consolidation of data from operations worldwide.
Support implementation of remote sensing and satellite imagery, artificial intelligence (AI) and machine learning in coordination with other humanitarian actors, by putting in place a methodology and infrastructure to utilize Unmanned Aerial Vehicles (UAV) for rapid response, risk reduction and planning in places without maps or in inaccessible locations.
In light of the COVID-19 pandemic, the Geospatial Analytics team has contributed to the global mobility database and online platform to support geospatial visualization of COVID-19 impacts on human mobility, across global, regional and country levels.

Priorities are ruled by ongoing emergencies but special attention for DTM’s geospatial teams goes to 1) Enhance the availability and use of geospatial data to achieve positive impacts for migrants and societies 2) Provide live feeds of migration trends and other relevant data into the UN Network Geospatial Data Hub 3) Strengthen evidence-base geospatial data on migration and displacement globally 4) Coordinate and share IOM efforts on geospatial analytics and methods with all UN agencies.

Composition of the unit:

The DTM GSA Team consist of four IM/GIS officers. The team works in close collaboration with DTM colleagues in HQ, regional offices, country missions and other IOM departments.

Contacts

For more information, please contact the DTM Data Management Team: [email protected].

For GIS support, please contact the DTM GSA team at [email protected].