The Potentials of Google Vision API-based Networks to Study Natively Digital Images

Article Sidebar

PDF (English) PDF (Español) (Español (España)) Fig. 11 a1 Fig. 11 a2
Published: Sep 1, 2021
DOI: https://doi.org/10.7764/disena.19.Article.1
Keywords:
Análisis de imágenes, Redes, Métodos digitales, Metodologías visuales, Visión artificial Image analysis, Networks, Digital methods, Visual methodologies, Computer vision

Main Article Content

Janna Joceli Omena
Center for Advanced Internet Studies (CAIS)
Pilipets Elena
Universität Klagenfurt, Department of Media and Communications
Beatrice Gobbo
Politecnico di Milano, DensityDesign Lab
Chao Jason
Universität Siegen, Collaborative Research Centre "Media of Cooperation"

Abstract

In this article, we present the potentials of Google Vision API-based networks for studying online images, covering three important modalities as part of a critical visual methodology: the content of the image itself, its specific ‘audiencing’ through web references (or image metadata), and the sites of image circulation. First, we conceptually and technically define different networks built upon computer vision features: image-label, image-web entities, and image-domain. Second, we present a research protocol diagram that illustrates how to build networks of images and respective descriptions or sites of circulation. Third, we discuss the potentialities of computer vision networks as a research device, stressing their data-relational (trans)formations and interpretative specifics. Three different case studies will be introduced as examples. In conclusion, we argue that such a visual methodology requires critical technical practices accounting for the multiple layers of technical mediation involved.


Downloads

Download data is not yet available.

Article Details

How to Cite
Omena, J. J., Elena , P., Gobbo, B. ., & Jason , C. (2021). The Potentials of Google Vision API-based Networks to Study Natively Digital Images. Diseña, (19), Article.1. https://doi.org/10.7764/disena.19.Article.1
Issue
No. 19 (2021): Visual Methods for Online Images: Collection, Circulation, and Machine Co-Creation
Section
Original Articles (part 1)

Licencia de Creative Commons
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International license.

COPYRIGHT NOTICE

All contents of this electronic edition are distributed under the Creative Commons license of "Attribution-ShareAlike 4.0 Internacional" (CC-BY-SA). Any total or partial reproduction of the material must mention its origin.

The rights of the published images belong to their authors, who grant to Diseña the license for its use. The management of the permits and the authorization of the publication of the images (or of any material) that contains copyright and its consequent rights of reproduction in this publication is the sole responsibility of the authors of the articles.

Author Biographies

Janna Joceli Omena, Center for Advanced Internet Studies (CAIS)

Master in Contemporary Culture and New Technologies, Universidade NOVA de Lisboa. Research fellow at the Center for Advanced Internet Studies (CAIS). She is a member of iNOVA Media Lab and the Public Data Lab. Her research focuses on digital methods, digital network studies, and technicity-of-the-mediums in support of social and medium research. She is the editor of Métodos Digitais: Teoria-Prática-Crítica (ICNOVA, 2019) and the coordinator of the SMART Data Sprint. Some of her latest publications are: ‘Digital Methods for Hashtag Engagement Research’ (with E.T. Rabello and A.G. Mintz; Social Media + Society, Vol. 6, Issue 3) and ‘Call into the Platform!’ (with A. Granado; Icono14, Vol. 18, Issue 1).

Pilipets Elena , Universität Klagenfurt, Department of Media and Communications

Doctor in Media Studies, Universität Klagenfurt. Postdoc researcher at the Depart­ment of Media and Communications, Universität Klagenfurt, and SMART (Social Media Research Techniques) researcher with iNOVA Media Lab, Universidade NOVA de Lisboa. Her teaching and research interests are related to media cultural studies, internet research, and digital methods. She is the current holder of the 2020 Carinthian Award for Young Social Sciences and Humanities Scholars. Her recent publications include ‘Nipples, Memes, and Algorithmic Failure: NSFW Critique of Tumblr Censorship’ (with S. Paasonen; New Media & Society, 1st Publ., 2020) and ‘Digitale Medien und Methoden. Über den methodischen Umgang mit visuellen Pla­ttforminhalten und Internet-Memes’ (Open-Media- Studies-Blog der Zeitschrift für Medienwissenschaft).

Beatrice Gobbo, Politecnico di Milano, DensityDesign Lab

Master in Communication Design, Politecnico di Milano. Ph.D. student in Design at Politecnico di Milano. She is a member of the Density­Design Lab, a research group focused on information visualization and information design. Her current research is focused on the role of communication design and information visualization in the field of explainable artificial intelligence. Some of her recent publications include ‘Research Protocol Diagrams as Didactic Tools to Act Critically in Dataset Design Pro­cesses’ (with M. Mauri, M. A. Briones, and G. Colom­bo; in INTED2020 Proceedings) and ‘Explaining AI through Critical Reflection Artifacts. On the Role of Communication Design within XAI’ (in T. Reis, M.X. Bornschlegl, M. Angelini, and M.L. Hemmje, Eds.; Advanced Visual Interfaces. Supporting Artificial Inte­lligence and Big Data Applications; Springer, 2020).

Chao Jason , Universität Siegen, Collaborative Research Centre "Media of Cooperation"

Master in Big Data and Digital Futu­res, University of Warwick. Master in Human Rights Law, University of London. He is a researcher and PhD candidate at the Universität Siegen. His current research is focused on the development of digital methods tools for the study of sensor media. He has backgrounds in software development and human rights advocacy. Some research software recently developed by him include: AppTraffic, to study the network traffic of mobile applications; and Offline Image Query and Extraction Tool (with J. J. Omena).

References

Azar, M., Cox, G., & Impett, L. (2021). Introduction: Ways of Machine Seeing. AI & Society. https://doi.org/10.1007/s00146-020-01124-6

Barreto, M. L., Barral-Netto, M., Stabeli, R., Almeida-Filho, N., Vasconcelos, P. F. C., Teixeira, M., Buss, P., & Gadelha, P. E. (2016). Zika Virus and Microcephaly in Brazil: A Scientific Agenda. The Lancet, 387(10022), 919–921. https://doi.org/10.1016/S0140-6736(16)00545-6

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An Open Source Software for Exploring and Manipulating Networks. Proceedings of the International AAAI Conference on Web and Social Media, 3(1), 361–362. https://doi.org/10.1136/qshc.2004.010033

Berners-Lee, T. (1995). Hypertext and Our Collective Destiny [Video]. 1995 Vannevar Bush Symposium - Videos and Links - Doug Engelbart Institute. Retrieved September 20, 2020, from https://www.dougengelbart.org/content/view/258/000/

Chao, J. (2021). Memespector GUI: Graphical User Interface Client for Computer Vision APIs [Computer Software] (Version 0.1) [C#] (Original work published 2021). https://github.com/jason-chao/memespector-gui

Colombo, G. (2018). The Design of Composite Images. Displaying Digital Visual Content for Social Research [Doctoral Dissertation, Politecnico di Milano]. http://hdl.handle.net/10589/141266

d’Andréa, C., & Mintz, A. (2019). Studying the Live Cross-Platform Circulation of Images with Computer Vision API: An Experiment Based on a Sports Media Event. International Journal of Communication, 13, 1825–1845.

Garimella, K., & Eckles, D. (2020). Images and Misinformation in Political Groups: Evidence from WhatsApp in India. Harvard Kennedy School Misinformation Review, 1(5). https://doi.org/10.37016/mr-2020-030

Geboers, M. A., & Van De Wiele, C. T. (2020). Machine Vision and Social Media Images: Why Hashtags Matter. Social Media + Society, 6(2), 2056305120928485. https://doi.org/10.1177/2056305120928485

Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2020). Datasheets for Datasets. ArXiv, 1803.09010 [cs].

Gerlitz, C., & Rieder, B. (2018). Tweets Are Not Created Equal: Investigating Twitter’s Client Ecosystem. International Journal of Communication, 12. https://hdl.handle.net/11245.1/4da1d406-1213-4103-8237-eef5ae786948

Gibbs, M., Meese, J., Arnold, M., Nansen, B., & Carter, M. (2015). #Funeral and Instagram: Death, Social Media, and Platform Vernacular. Information, Communication & Society, 18(3), 255–268. https://doi.org/10.1080/1369118X.2014.987152

Google Cloud. (2017, April 13). Pricing | Google Cloud Vision API Documentation. Wayback Machine. https://web.archive.org/web/20170413081619/https://cloud.google.com/vision/docs/pricing

Google User Content. (2020). General Guidelines. https://static.googleusercontent.com/media/guidelines.raterhub.com/pt-BR//searchqualityevaluatorguidelines.pdf

Hochman, N. (2014). The Social Media Image. Big Data & Society, 1(2), 2053951714546645. https://doi.org/10.1177/2053951714546645

Hoelzl, I., & Marie, R. (2015). Softimage: Towards a New Theory of the Digital Image. Intellect.

Jacomy, M. (2013). Table 2 Net [Computer Software]. https://medialab.github.io/table2net/

Jacomy, M. (2019). The Web as Layers [Video]. HYPHE - Session 4. https://panopto.aau.dk/Panopto/Pages/Viewer.aspx?id=48cfe5ff-5503-431b-887f-ab53007ef5c4

Jacomy, M. (2021). Situating Visual Network Analysis [Doctoral Dissertation, Aalborg University]. https://reticular.hypotheses.org/1879

Jacomy, M., Venturini, T., Heymann, S., & Bastian, M. (2014). ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software. PLOS ONE, 9(6), e98679. https://doi.org/10.1371/journal.pone.0098679

Leetaru, K. (2019, June 1). Using Google Vision AI’s Reverse Image Search To Richly Catalog Television News. Forbes. https://www.forbes.com/sites/kalevleetaru/2019/06/01/using-google-vision-ais-reverse-image-search-to-richly-catalog-television-news/

MacKenzie, A., & Munster, A. (2019). Platform Seeing: Image Ensembles and their Invisualities. Theory, Culture & Society, 36(5), 3–22. https://doi.org/10.1177/0263276419847508

Manovich, L. (2020). Cultural Analytics. MIT Press.

Mauri, M., Briones, M. A., Gobbo, B., & Colombo, G. (2020). Research Protocol Diagrams as Didactic Tools to Act Critically in Dataset Design Processes. INTED2020 Proceedings, 9034–9043. https://library.iated.org/view/MAURI2020RES

Mintz, A., Silva, T., Gobbo, B., Pilipets, E., Azhar, H., Takamitsu, H., Omena, J. J., & Oliveira, T. (2019). Interrogating Vision APIs. #SMARTDataSprint. https://smart.inovamedialab.org/past-editions/smart-2019/project-reports/interrogating-vision-apis/

Niederer, S., & Colombo, G. (2019). Visual Methodologies for Networked Images: Designing Visualizations for Collaborative Research, Cross-platform Analysis, and Public Participation. Diseña, (14), 40–67. https://doi.org/10.7764/disena.14.40-67

Omena, J. J. (in press). Digital Methods and Technicity-of-the-Mediums. From Regimes of Functioning to Digital Research [Doctoral Dissertation]. Universidade NOVA de Lisboa.

Omena, J. J., & Amaral, I. (2019). Sistema de leitura de redes digitais multiplataforma. In J. J. Omena (Ed.), Métodos Digitais: Teoria‐prática‐crítica (pp. 121–140). ICNOVA.

Omena, J. J., & Granado, A. (2020). Call into the Platform! Revista ICONO14 Revista Científica de Comunicación y Tecnologías emergentes, 18(1), 89–122. https://doi.org/10.7195/ri14.v18i1.1436

Omena, J. J., Rabello, E. T., & Mintz, A. G. (2020). Digital Methods for Hashtag Engagement Research. Social Media + Society, 6(3), 2056305120940697. https://doi.org/10.1177/2056305120940697

Paglen, T. (2014, March 13). Seeing Machines. Is Photography Over? Fotomuseum Winterthur Series. https://www.fotomuseum.ch/en/2014/03/13/seeing-machines/

Parikka, J., & Dvořák, T. (2021). Introduction: On the Scale, Quantity and Measure of Images. In T. Dvořák & J. Parikka (Eds.), Photography Off the Scale: Technologies and Theories of the Mass Image (pp. 1–24). Edinburgh University Press.

Pearce, W., Özkula, S. M., Greene, A. K., Teeling, L., Bansard, J. S., Omena, J. J., & Rabello, E. T. (2020). Visual Cross-platform Analysis: Digital Methods to Research Social Media Images. Information, Communication & Society, 23(2), 161–180. https://doi.org/10.1080/1369118X.2018.1486871

Pilipets, E., Flores, A. M. M., Flaim, G., Skazedonig, M., Sepúlveda, R., & Del Nero, S. (2020). From “Tumblr Purge” to “Female Nipples”: Telling a Story of Platform Censorship Critique through Memes and Digital Methods. #SMARTDataSprint. https://smart.inovamedialab.org/2020-digital-methods/project-reports/tumblr-purge-female-nipples/

Pilipets, E., & Paasonen, S. (2020). Nipples, Memes, and Algorithmic Failure: NSFW Critique of Tumblr Censorship. New Media & Society, 1461444820979280. https://doi.org/10.1177/1461444820979280

Rettberg, J. W. (2020). Situated Data Analysis: A New Method for Analysing Encoded Power Relationships in Social Media Platforms and Apps. Humanities and Social Sciences Communications, 7, Art. 5. https://doi.org/10.1057/s41599-020-0495-3

Ricci, D., Colombo, G., Meunier, A., & Brilli, A. (2017). Designing Digital Methods to Monitor and Inform Urban Policy: The Case of Paris and its Urban Nature Initiative. International Conference on Public Policy (ICPP3). https://hal.archives-ouvertes.fr/hal-01903809

Rieder, B. (2015). YouTube Data Tools [Computer Software] (Version 1.11). https://tools.digitalmethods.net/netvizz/youtube/

Rieder, B. (2020). Engines of Order: A Mechanology of Algorithmic Techniques. Amsterdam University Press.

Rieder, B., & Röhle, T. (2018). Digital Methods: From Challenges to Bildung. In M. T. Schäfer & K. van Es (Eds.), The Datafied Society: Studying Culture through Data (pp. 109–124). Amsterdam University Press. https://doi.org/10.1515/9789048531011-010

Robinson, S. (2017, June 1). Exploring the Cloud Vision API. Medium. https://medium.com/@srobtweets/exploring-the-cloud-vision-api-1af9bcf080b8

Rogers, R. (2013). Digital Methods. MIT Press.

Rogers, R. (2019). Doing Digital Methods. SAGE.

Rose, G. (2016). Visual Methodologies (4th ed.). Open University.

Rubinstein, D., & Sluis, K. (2013). Concerning the Undecidability of the Digital Image. Photographies, 6(1), 151–158. https://doi.org/10.1080/17540763.2013.788848

Schwemmer, C., Knight, C., Bello-Pardo, E. D., Oklobdzija, S., Schoonvelde, M., & Lockhart, J. W. (2020). Diagnosing Gender Bias in Image Recognition Systems. Socius, 6, 2378023120967171. https://doi.org/10.1177/2378023120967171

Silva, T., Barciela, P., & Meirelles, P. (2018). Mapeando Imagens de Desinformação e Fake News Político-Eleitorais com Inteligência Artificial. 3o CONEC: Congresso Nacional de Estudos Comunicacionais Da PUC Minas Poços de Caldas - Convergência e Monitoramento, 413–427. https://www.researchgate.net/publication/329525177_Mapeando_Imagens_de_Desinformacao_e_Fake_News_Politico-Eleitorais_com_Inteligencia_Artificial

Silva, T., Mintz, A., Omena, J. J., Gobbo, B., Oliveira, T., Takamitsu, H. T., Pilipets, E., & Azhar, H. (2020). APIs de Visão Computacional: Investigando mediações algorítmicas a partir de estudo de bancos de imagens. Logos, 27(1), 25–54. https://doi.org/10.12957/logos.2020.51523

Steyerl, H. (2009). In Defense of the Poor Image. E-Flux Journal, 10. https://www.e-flux.com/journal/10/61362/in-defense-of-the-poor-image/

Sullivan, D. (2020, May 20). A Reintroduction to our Knowledge Graph and Knowledge Panels. Blog.Google. https://blog.google/products/search/about-knowledge-graph-and-knowledge-panels/

Szeliski, R. (2021). Computer Vision: Algorithms and Applications (2nd ed.). Springer.

Taibi, D., Rogers, R., Marenzi, I., Nejdl, W., Ahmad, Q. A. I., & Fulantelli, G. (2016). Search as Research Practices on the Web: The Sar-web Platform for Cross-language Engine Results Analysis. Proceedings of the 8th ACM Conference on Web Science, 367–369. https://doi.org/10.1145/2908131.2908201

Tumblr. (2018, December). Updates to Tumblr’s Community Guidelines [Tumblr]. Tumblr Support. https://support.tumblr.com/post/180758979032/updates-to-tumblrs-community-guidelines

Venturini, T., Jacomy, M., & Jensen, P. (2019). What Do We See When We Look at Networks. An Introduction to Visual Network Analysis and Force-Directed Layouts (SSRN Scholarly Paper ID 3378438). Social Science Research Network. https://doi.org/10.2139/ssrn.3378438

Venturini, T., Jacomy, M., & Pereira, D. (2015). Visual Network Analysis: The Example of the Rio+20 Online Debate. https://hal.archives-ouvertes.fr/hal-02305124

Xi, N., Ma, D., Liou, M., Steinert-Threlkeld, Z. C., Anastasopoulos, J., & Joo, J. (2020). Understanding the Political Ideology of Legislators from Social Media Images. Proceedings of the 14th International AAAI Conference on Web and Social Media, ICWSM 2020, 726–737. http://arxiv.org/abs/1907.09594