publications

2023

1. Prompting in Autoregressive Large Language Models

   Prabin Bhandari

   arXiv preprint arXiv:2312.03740, 2023

   Abs Bib PDF

   Autoregressive Large Language Models have transformed the landscape of Natural Language Processing. Pre-train and prompt paradigm has replaced the conventional approach of pre-training and fine-tuning for many downstream NLP tasks. This shift has been possible largely due to LLMs and innovative prompting techniques. LLMs have shown great promise for a variety of downstream tasks owing to their vast parameters and huge datasets that they are pre-trained on. However, in order to fully realize their potential, their outputs must be guided towards the desired outcomes. Prompting, in which a specific input or instruction is provided to guide the LLMs toward the intended output, has become a tool for achieving this goal. In this paper, we discuss the various prompting techniques that have been applied to fully harness the power of LLMs. We present a taxonomy of existing literature on prompting techniques and provide a concise survey based on this taxonomy. Further, we identify some open problems in the realm of prompting in autoregressive LLMs which could serve as a direction for future research.

   @article{bhandari2023prompting,
     title = {Prompting in Autoregressive Large Language Models},
     author = {Bhandari, Prabin},
     journal = {arXiv preprint arXiv:2312.03740},
     year = {2023},
   }

2. Are Large Language Models Geospatially Knowledgeable?

   Prabin Bhandari, Antonios Anastasopoulos, and Dieter Pfoser

   arXiv preprint arXiv:2310.13002, 2023

   Abs Bib PDF

   Despite the impressive performance of Large Language Models (LLM) for various natural language processing tasks, little is known about their comprehension of geographic data and related ability to facilitate informed geospatial decision-making. This paper investigates the extent of geospatial knowledge, awareness, and reasoning abilities encoded within such pretrained LLMs. With a focus on autoregressive language models, we devise experimental approaches related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge, (ii) using geospatial and non-geospatial prepositions to gauge their geospatial awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to assess the models’ geospatial reasoning capabilities and to determine locations of cities based on prompting. Our results confirm that it does not only take larger, but also more sophisticated LLMs to synthesize geospatial knowledge from textual information. As such, this research contributes to understanding the potential and limitations of LLMs in dealing with geospatial information.

   @article{bhandari2023large,
     title = {Are Large Language Models Geospatially Knowledgeable?},
     author = {Bhandari, Prabin and Anastasopoulos, Antonios and Pfoser, Dieter},
     journal = {arXiv preprint arXiv:2310.13002},
     year = {2023},
   }

3. Trustworthiness of Children Stories Generated by Large Language Models

   Prabin Bhandari, and Hannah Brennan

   In Proceedings of the 16th International Natural Language Generation Conference, Sep 2023

   Abs Bib PDF

   Large Language Models (LLMs) have shown a tremendous capacity for generating literary text. However, their effectiveness in generating children’s stories has yet to be thoroughly examined. In this study, we evaluate the trustworthiness of children’s stories generated by LLMs using various measures, and we compare and contrast our results with both old and new children’s stories to better assess their significance. Our findings suggest that LLMs still struggle to generate children’s stories at the level of quality and nuance found in actual stories.

   @inproceedings{bhandari-brennan-2023-trustworthiness,
     title = {Trustworthiness of Children Stories Generated by Large Language Models},
     author = {Bhandari, Prabin and Brennan, Hannah},
     booktitle = {Proceedings of the 16th International Natural Language Generation Conference},
     month = sep,
     year = {2023},
     address = {Prague, Czechia},
     publisher = {Association for Computational Linguistics},
     pages = {352--361},
   }

4. Estimation of Vehicular Velocity based on Non-Intrusive stereo camera

   Bikram Adhikari, and Prabin Bhandari

   arXiv preprint arXiv:2304.05298, Sep 2023

   Abs Bib PDF

   The paper presents a modular approach for the estimation of a leading vehicle’s velocity based on a non-intrusive stereo camera where SiamMask is used for leading vehicle tracking, Kernel Density estimate (KDE) is used to smooth the distance prediction from a disparity map, and LightGBM is used for leading vehicle velocity estimation. Our approach yields an RMSE of 0.416 which outperforms the baseline RMSE of 0.582 for the SUBARU Image Recognition Challenge

   @article{adhikari2023estimation,
     title = {Estimation of Vehicular Velocity based on Non-Intrusive stereo camera},
     author = {Adhikari, Bikram and Bhandari, Prabin},
     journal = {arXiv preprint arXiv:2304.05298},
     year = {2023},
   }