Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve org matching #78

Closed
wants to merge 3 commits into from
Closed

Improve org matching #78

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
40741c4
Improve org matching
b-j-mills Apr 16, 2024
ba76031
Revert hapi-schema version
b-j-mills Apr 16, 2024
3daea40
Revert all requirements
b-j-mills Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions src/hapi/pipelines/database/operational_presence.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ def populate(self, debug=False):
admin_code
)
if admin_level == "adminone":
country_code = self._admintwo.pcode_to_iso3.get(
country_code = self._adminone.pcode_to_iso3.get(
admin_code
)
org_info = self._org.get_org_info(
Expand Down Expand Up @@ -138,7 +138,6 @@ def populate(self, debug=False):
and (
clean_name(org_acronym).upper(),
clean_name(org_name),
org_type_code,
)
not in self._org.data
):
Expand All @@ -152,6 +151,7 @@ def populate(self, debug=False):
sector_code = self._sector.get_sector_code(sector_orig)
if debug:
debug_row = {
"location": country_code,
"org_name_orig": org_name_orig,
"org_acronym_orig": org_acronym_orig,
"org_type_orig": org_type_orig,
Expand All @@ -175,7 +175,6 @@ def populate(self, debug=False):
(
clean_name(org_acronym).upper(),
clean_name(org_name),
org_type_code,
)
]
admin2_ref = self._admins.admin2_data[admin2_code]
Expand Down
6 changes: 4 additions & 2 deletions src/hapi/pipelines/database/org.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ def populate(self):
canonical_org_name = row.get("#org+name")
if canonical_org_name:
self._org_map[canonical_org_name] = row
org_acronym = row.get("#org+acronym")
if org_acronym:
self._org_map[org_acronym] = row

def populate_single(
self,
Expand All @@ -61,14 +64,13 @@ def populate_single(
self._session.add(org_row)
self._session.commit()
results = self._session.execute(
select(DBOrg.id, DBOrg.acronym, DBOrg.name, DBOrg.org_type_code)
select(DBOrg.id, DBOrg.acronym, DBOrg.name)
)
for result in results:
self.data[
(
clean_name(result[1]).upper(),
clean_name(result[2]),
result[3],
)
] = result[0]

Expand Down
110 changes: 87 additions & 23 deletions ...dzgev9-a6yu7vtcewhblm56aul1iifrfv5ubvvjvt7b-pub-gid-1040329566-single-true-output-csv.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,25 @@
String pattern,Canonical org name,Acronym,Org type,Location filter,Sector filter,Org type code,Location code,Sector code
#x_pattern,#org+name,#org+acronym,#org+type,#country+name,#sector+name,#org+type+code,#country+code,#sector+code
Fonds des Nations Unies pour l'Enfance,United Nations Children's Fund,UNICEF,United Nations,,,447,,
United Nations Children's Fund,United Nations Children's Fund,UNICEF,United Nations,,,447,,
United Nations Children's Emergency Fund,United Nations Children's Fund,UNICEF,United Nations,,,447,,
Fond des Nations Unies pour l'Enfance,United Nations Children's Fund,UNICEF,United Nations,,,447,,
United Nations International Childrens Emergency Fund,United Nations Children's Fund,UNICEF,United Nations,,,447,,
UNICEF - Fondo de las Naciones Unidas para la Infancia,United Nations Children's Fund,UNICEF,United Nations,,,447,,
World Health Organization,World Health Organization,WHO,United Nations,,,447,,
Organisation Mondiale de la Santé,World Health Organization,WHO,United Nations,,,447,,
International Rescue Committee,International Rescue Committee,IRC,International NGO,,,437,,
International Rescue Comity,International Rescue Committee,IRC,International NGO,,,437,,
Programme Alimentaire Mondial,World Food Programme,WFP,United Nations,,,447,,
Population Service International,Population Services International,PSI,International NGO,,,437,,
Catholic Relief Service,Catholic Relief Services,CRS,International NGO,,,437,,
United Nations Children's Fund,United Nations Children's Fund,UNICEF,United Nations,,,447,,
Catholic Relief Services,Catholic Relief Services,CRS,International NGO,,,437,,
Norwegian Refugee Council,Norwegian Refugee Council,NRC,International NGO,,,437,,
International Organization for Migration,International Organization for Migration,IOM,United Nations,,,447,,
Danish Refugee Council,Danish Refugee Council,DRC,International NGO,,,437,,
ONG MUSO,Muso,,National NGO,Mali,,441,MLI,
United Nations Children's Emergency Fund,United Nations Children's Fund,UNICEF,United Nations,,,447,,
Marie Stopes International,Marie Stopes International,MSI,International NGO,,,437,,
Association Malienne pour la Protection et le Développement de l'Environnement du Sahel,Association Malienne pour la Protection de l'Environnement,STOP-SAHEL,National NGO,,,441,,
Fonds des Nations Unies pour la Population,United Nations Population Fund,UNFP,United Nations,,,447,,
Agency For Technical Cooperation & Development,,,,,,,,
Action Contre la Faim-Espagne,Action against Hunger - Spain,ACF Spain,International NGO,,,437,,
Save the Children Federation International,Save the Children International,SCI,International NGO,,,437,,
Fonds des Nations Unies pour la Population,United Nations Population Fund,UNFPA,United Nations,,,447,,
UNFPA - United Nations Population Fund,United Nations Population Fund,UNFPA,United Nations,,,447,,
Association Malienne pour la Survie au Sahel,,,,,,,,
Winrock International,,,,,,,,
Œuvre Malienne d'Aide à l'Enfance du Sahel,,,,,,,,
Expand All @@ -27,37 +28,54 @@ World Education Inc.,,,,,,,,
Global Health Supply Chain-Procurement and Supply Management,,,,,,,,
Plan International,Plan International,PI,International NGO,,,437,,
Comité International de la Croix Rouge,International Committee of the Red Cross,ICRC,Red Cross / Red Crescent,,,445,,
International Organization For Migration,International Organization for Migration,IOM,United Nations,,,447,,
International Organization for Migration,International Organization for Migration,IOM,United Nations,,,447,,
International Organisation for Migrations,International Organization for Migration,IOM,United Nations,,,447,,
INTERNATIONALE ORGANISATION FOR MIGRATION,International Organization for Migration,IOM,United Nations,,,447,,
Organisation Internationale des Migrations,International Organization for Migration,IOM,United Nations,,,447,,
OIM - International Organization for Migration,International Organization for Migration,IOM,United Nations,,,447,,
Organisation Internationale pour les Migrations,International Organization for Migration,IOM,United Nations,,,447,,
International Medical Corps,International Medical Corps,IMC,International NGO,,,437,,
Organization for Community Coordination of Development,,,,,,,,
Organisation des Nations Unies pour l'Alimentation et l'Agriculture,Food and Agriculture Organization,FAO,United Nations,,,447,,
Family Health International,,,,,,,,
FAO - Food & Agriculture Organization of the United Nations,Food and Agriculture Organization,FAO,United Nations,,,447,,
Fond des Nations Unies pour l'Agriculture,Food and Agriculture Organization,FAO,United Nations,,,447,,
Food & Agriculture Organization of the United Nations,Food and Agriculture Organization,FAO,United Nations,,,447,,
Food Agricultural organization,Food and Agriculture Organization,FAO,United Nations,,,447,,
Food and Agriculture Organization of the United Nations,Food and Agriculture Organization,FAO,United Nations,,,447,,
Food and Agriculture Organization,Food and Agriculture Organization,FAO,United Nations,,,447,,
Food and Agriculture Organisation,Food and Agriculture Organization,FAO,United Nations,,,447,,
Family Health International,,,,,,,,
Women for Afghan Women,,,,,,,,
Save the Children Federation International,Save the Children International,SCI,International NGO,,,437,,
Save The Children International,Save the Children International,SCI,International NGO,,,437,,
Save the Children,Save the Children International,SCI,International NGO,,,437,,
SC - Save the Children,Save the Children International,SCI,International NGO,,,437,,
Action Contre la Faim-Espagne,Action against Hunger - Spain,ACF Spain,International NGO,,,437,,
Acción Contra el Hambre,Action against Hunger,ACF,International NGO,,,437,,
ACH,Action against Hunger,ACF,International NGO,,,437,,
Action Against Hunger,Action against Hunger,ACF,International NGO,,,437,,
Action Contre la Faim,Action against Hunger,ACF,International NGO,,,437,,
Action Against Hunger | ACF-USA,Action against Hunger - US,ACF - US,International NGO,,,437,,
Croix Rouge Malienne,Mali Red Cross,,National NGO,,,441,,
World Food Program,World Food Programme,WFP,United Nations,,,447,,
World Vision International,World Vision International,WVI,International NGO,,,437,,
Save the Children International,Save the Children International,SCI,International NGO,,,437,,
World Vision,World Vision International,WVI,International NGO,,,437,,
World Vision Interationale,World Vision International,WVI,International NGO,,,437,,
Just for Afghan Capacity and Knowledge,,,,,,,,
Association Malienne pour la Protection et la Promotion de la Famille,,,,,,,,
Association Conseil pour le développement,,,,,,,,
Swedish Committee for Afghanistan,,,,,,,,
Action Education,Action Education,,International NGO,,,437,,
Coordination of Afghan Relief,,,,,,,,
Première Urgence-Aide Medicale Internationale,,,,,,,,
United Nations Population Fund,United Nations Population Fund,UNFP,United Nations,,,447,,
Fundación Educación y Cooperación,,,,,,,,
Care International in Afghanistan,,,,,,,,
CARE International,CARE International,CARE,International NGO,,,437,,
Association de Soutien au Développement des Activités de Population,,,,,,,,
Intersos Humanitarian Aid,,,,,,,,
United Nations High Commissioner for Refugees,UN Refugee Agency,UNHCR,United Nations,,,447,,
United Nations High Commisionner for Refugees,UN Refugee Agency,UNHCR,United Nations,,,447,,
Haut Commissariat des Nations Unies pour les Réfugiés,UN Refugee Agency,UNHCR,United Nations,,,447,,
United Nations High Commisionner for Refugees,UN Refugee Agency,UNHCR,United Nations,,,447,,
IDS-Hamkar,,,,,,,,
INTERNATIONAL MEDICAL CORPS,International Medical Corps,IMC,International NGO,,,437,,
World Vision Interationale,World Vision International,WVI,International NGO,,,437,,
Houseshold and Community Health,,,,,,,,
Adamawa State Primary Health Care Management Board,,,,,,,,
Mercy Corps,Mercy Corps International,MCI,International NGO,,,437,,
Expand All @@ -71,6 +89,8 @@ Goal Prime Organisation Nigeria,,,,,,,,
Organization of Health and Program Management,,,,,,,,
Afghan Amputee Bicyclist for Rehabilitation and Recreation,,,,,,,,
Bureau des Nations Unies pour la Coordination des Affaires Humanitaires,United Nations Office for the Coordination of Humanitarian Affairs,OCHA,United Nations,,,447,,
Bureau des Nation Unies pour la Coordination des Affaires Humanitaires,United Nations Office for the Coordination of Humanitarian Affairs,OCHA,United Nations,,,447,,
Bureau de la coordination des affaires humanitaires,United Nations Office for the Coordination of Humanitarian Affairs,OCHA,United Nations,,,447,,
Health Net - Trans cultural Psychosocial Organization,,,,,,,,
National Data Archive Centre,,,,,,,,
State Ministry Of Health,,,,,,,,
Expand All @@ -80,16 +100,13 @@ Association pour l’Appui aux Populations rurales du Mali,,,,,,,,
Yobe State Primary Health Care Management Board,,,,,,,,
Agency for Assistance and Development of Afghanistan,,,,,,,,
Life Saving Grassroots Outreach,,,,,,,,
Catholic Relief Services,Catholic Relief Services,CRS,International NGO,,,437,,
CENTRE CARTER,,,,,,,,
Delta Survie,,,,,,,,
YEUX DU MONDE,,,,,,,,
Bakhtar Development Network,,,,,,,,
Première Urgence Internationale,,,,,,,,
Islamic Relief,Islamic Relief Worldwide,IR,International NGO,,,437,,
Afghanistan Development & Welfare Services Organization,,,,,,,,
Norwegian Afghanistan Committee,,,,,,,,
Action Against Hunger | ACF-USA,Action against Hunger - US,ACF - US,International NGO,,,437,,
Children in Crisis,,,,,,,,
Association Malienne pour le Developpement Communautaire,,,,,,,,
Médecins du Monde,,,,,,,,
Expand All @@ -102,17 +119,21 @@ Organization for Coordination of Humanitarian Relief,Organization for Coordinati
Watan Social and Technical Services Association,,,,,,,,
Helen Keller International,,,,,,,,
Humanitarian Assistance for Development of Afghanistan,,,,,,,,
CARE International,CARE International,CARE,International NGO,,,437,,
Organization for Relief and Development,,,,,,,,
Welthungerhilfe,,,,,,,,
Cooperazione Internazionale,,,,,,,,
Welfare Association for Development of Afghanistan,,,,,,,,
Aga Khan Health Service,,,,,,,,
Federal Neuro-Psychiatric Hospital,,,,,,,,
Solidarites International,,,,,,,,
Médecins Sans Frontière,Doctors without Borders,MSF,International NGO,,,437,,
Medecins Sans Frontieres (Doctors Without Borders),Doctors without Borders,MSF,International NGO,,,437,,
Médecins Sans Frontière-France,Doctors Without Borders France,MSF-F,International NGO,,,437,,
Médecins sans frontières France,Doctors Without Borders France,MSF-F,International NGO,,,437,,
Médecins sans frontières Hollande,Doctors Without Borders Holland,MSF-H,International NGO,,,437,,
Médecins sans frontières Holland,Doctors Without Borders Holland,MSF-H,International NGO,,,437,,
Médecins sans frontières Switezerland,Doctors Without Borders Switzerland,MSF-CH,International NGO,,,437,,
Médecins sans frontières Suisse,Doctors Without Borders Switzerland,MSF-CH,International NGO,,,437,,
Green Concern For Development,,,,,,,,
Intersos - Humanitarian Organization for Emergency,,,,,,,,
Royal Heritage Health Foundation,,,,,,,,
SIGHT SAVERS INTERNATIONAL,,,,,,,,
Catholic Organization for Relief and Development Aid,,,,,,,,
Expand All @@ -127,4 +148,47 @@ Association Malienne pour la Solidarité et le Développement,,,,,,,,
Care Best Initiative,,,,,,,,
AECID,,,,,,,,
Norwegian Project Office / Rural Rehabilitation Association for Afghanistan,,,,,,,,
Social and Humanitarian Assistance Organization,,,,,,,,
Social and Humanitarian Assistance Organization,,,,,,,,
Intersos Humanitarian Aid,INTERSOS - Humanitarian Aid Organization,INTERSOS,International NGO,,,437,,
Intersos - Humanitarian Organization for Emergency,INTERSOS - Humanitarian Aid Organization,INTERSOS,International NGO,,,437,,
Humanitarian Aid Organization,INTERSOS - Humanitarian Aid Organization,INTERSOS,International NGO,,,437,,
Intersos Humanitarian Aid Organization,INTERSOS - Humanitarian Aid Organization,INTERSOS,International NGO,,,437,,
Doctors of the World,Médecins du Monde,MDM,International NGO,,,437,,
Médecins du Monde,Médecins du Monde,MDM,International NGO,,,437,,
Agency for Technical Cooperation and Development,Agency for Technical Cooperation and Development,ACTED,International NGO,,,437,,
Agency For Technical Cooperation & Development,Agency for Technical Cooperation and Development,ACTED,International NGO,,,437,,
ACTED - Agencia de Cooperación Técnica y Desarrollo,Agency for Technical Cooperation and Development,ACTED,International NGO,,,437,,
Agence d'Aide à la Coopération Technique et au Développement,Agency for Technical Cooperation and Development,ACTED,International NGO,,,437,,
"United Nations Educational, Scientific and Cultural Organization","United Nations Educational, Scientific and Cultural Organization",UNESCO,United Nations,,,447,,
"The United Nations Educational, Scientific and Cultural Organization","United Nations Educational, Scientific and Cultural Organization",UNESCO,United Nations,,,447,,
United Nations Mine Action Services,United Nations Mine Action Service,UNMAS,United Nations,,,447,,
United Nationsl Mine Action Service,United Nations Mine Action Service,UNMAS,United Nations,,,447,,
British Red Cross,British Red Cross,BRC,Red Cross / Red Crescent,,,445,,
International Federation of Red Cross and Red Crescent Societies,International Federation of Red Cross and Red Crescent Societies,IFRC,Red Cross / Red Crescent,,,445,,
Catholic Organization for relief Development and Aid,Catholic Organization for Relief and Development Aid,CORDAID,International NGO,,,437,,
Programme des Nations Unies pour le Développement,United Nations Development Programme,UNDP,United Nations,,,447,,
United Nations Human Settlements Programme,United Nations Human Settlements Programme,UN-Habitat,United Nations,,,447,,
United Nations Entity for Gender Equality and,United Nations Entity for Gender Equality and the Empowerment of Women,UN Women,United Nations,,,447,,
United Nations Department for Security and Safety,United Nations Department for Safety and Security,UNDSS,United Nations,,,447,,
CENTRE D'ETUDE ET COOPERATION INTERNATIONALE,Centre d'Etude et de Cooperation Internationale,CECI,International NGO,,,437,,
German Society for International Cooperation,German Development Cooperation,GIZ,International NGO,,,437,,
Cooperation Allemande,German Development Cooperation,GIZ,International NGO,,,437,,
Deutsche Gesellschaft für Internationale Zusammenarbeit,German Development Cooperation,GIZ,International NGO,,,437,,
Première Urgence Internationale,,,,,,,,
Premiere Urgence International,Première Urgence Internationale,PUI,International NGO,,,437,,
Première Urgence,Première Urgence Internationale,PUI,International NGO,,,437,,
PUI - Première Urgence Internationale,Première Urgence Internationale,PUI,International NGO,,,437,,
Premiere Urgence-Aide Medicale Internationale,Première Urgence Internationale,PUI,International NGO,,,437,,
Première Urgence - Aide Médicale Internationale,Première Urgence Internationale,PUI,International NGO,,,437,,
SOLIDARITÉS INTERNATIONALE,Solidarities International,SI,International NGO,,,437,,
Solidarites International,Solidarities International,SI,International NGO,,,437,,
WELTHUNGERHILFE/AGRO ACTION ALLEMANDE,Welthungerhilfe,WHH,International NGO,,,437,,
Deutsche Welthungerhilfe e. V. (German Agro Action),Welthungerhilfe,WHH,International NGO,,,437,,
AGENCE ADVENTISTE DE DÉVELOPPEMENT ET DE SECOURS,Adventist Development and Relief Agency,ADRA,International NGO,,,437,,
Family Health International,Family Health International,FHI360,International NGO,,,437,,
HelpCode,HelpCode,HelpCode,International NGO,,,437,,
Oxford Committee for Famine Relief GB,OXFAM,OXFAM,International NGO,,,437,,
Caritas Internationalis,Caritas,Caritas,International NGO,,,437,,
Street Child,Street Child,Street Child,International NGO,,,437,,
Handicap International,Handicap International - Humanity & Inclusion,HI,International NGO,,,437,,
Humanité & Inclusion,Handicap International - Humanity & Inclusion,HI,International NGO,,,437,,
Loading
Loading