Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Lavastorm Kerberos authentication through jdbc/odbc drivers to Hadoop

  1. #1

    Default Lavastorm Kerberos authentication through jdbc/odbc drivers to Hadoop

    Hi ,

    Got couple of questions

    We have got Hadoop as data source for most of our projects and on Hadoop we are implementing security authentication based on kerberos certificates

    Wanted to know whether LAE client/BRE and lavastorm server support kerberos based authentication ?
    Please note that we are running LAE on Linux server

    Appreciate your quick response on the same
    -Sreekanth

  2. #2
    Lavastorm Employee
    Join Date
    Nov 2012
    Location
    Warrington, UK
    Posts
    232

    Default

    Hi Sreekanth,

    at this time we do not support Kerberos authentication. However, this was already marked as an item on our roadmap.

    Are you intending to user both JDBC and ODBC or is one your prefered method?

    Regards,

    Adrian

  3. #3

    Default

    Hi ,
    We are presently planning use JDBC drivers.
    May I know do you have any documentation on installing ODBC drivers for Oracle , Hadoop on LAE 4.6 server running on Linux ?

    -Sreekath

  4. #4
    Lavastorm Employee
    Join Date
    Nov 2012
    Location
    Warrington, UK
    Posts
    232

    Default

    Hi Sreekanth,

    the ODBC driver installation and configuration information has now been sent to you by email.

    Adrian

  5. #5

    Default

    Hello Adrian,

    Do you have any driver available for JDBC so I can connect LAE to Hadoop? If not, can I please request some guidance on how to resolve the error "unable to load db driver: cloudera.impala.jdbc4.Driver"?

    Thanks in advance for your kind help!

    Regards,

    tris

  6. #6
    Lavastorm Employee
    Join Date
    Nov 2012
    Location
    Warrington, UK
    Posts
    232

    Default

    Hi tris,

    Lavastorm do not redistribute JDBC drivers for Hadoop platforms. You will need to obtain the drivers from the appropriate Hadoop distribution vendor's website, e.g.:

    Cloudera - https://www.cloudera.com/downloads.html

    Hortonworks - https://hortonworks.com/downloads/

    ** Upfront caveat: There are many issues that can prevent a successful JDBC connection to a Kerberos-secured Hadoop cluster. I do not have a secure cluster available to test this against so your mileage may vary on this ...

    The driver you are using has to be compatible with the Java version used by Lavastorm. Lavastorm 6.1 uses Java 7. Check the system requirements in the driver's installation guide.

    I have not used Impala but I have seen some issues with loading other JDBC drivers for Hadoop (specifically Hive) and the underlying cause may be the same. The driver typically comprise a number of jar files and there may be issues conflicts with the core jars used by Lavastorm if the driver jars have been copied into the lib/java/ext directory.

    This can be overcome by:
    a) Extract the driver files and copying them into an accessible directory on the machine hosting Lavastorm.

    b) Declare a text type property on the JDBC Query node named 'Classpaths' and put it in the 'Java' property group.

    c) Enter the file path to each of the driver's jars in the 'Classpaths' text property. Use a separate line for each path.
    If required you could use a graph run property to specify the directory portion of the file paths so that you can change
    this more easily if you want to run the data flow on different machines.

    An example of creating a node with the custom Classpaths is shown below.

    Hive_JDBC_Query_Node_with_Custom_Classpaths_Example.brg

    In my case the 'HiveDriverFP' property was set to the directory where the jars had been copied to e.g.
    C:/Program Files/Lavastorm/LAE6.1/ExternalDrivers/Cloudera_HiveJDBC41_2.5.16

    Check the driver's installation and configuration guide for the appropriate information required to configure the driver. The node in the example reflects connecting to a cluster that is using username authentication. You will need to change the configuration if you are trying to use Kerberos, and it will have to align with the method by which the driver obtains the TicketGrantingTicket and Kerberos Principal.

    In particular you will need to modify the connection string in the DbUrl property. e.g. (yours will probably be different depending on your driver version and Kerberos set-up, and may need more or fewer attributes)

    jdbc:hive2://hiveHostname:10000;AuthMech=1;KrbRealm=HIVEHOSTNAM E.LAVASTORM.COM;KrbHostFQDN=hiveHostname.lavastorm .com;KrbServiceName=hive
    Last edited by awilliams1024; 11-23-2017 at 03:34 PM.

  7. #7

    Default

    Thanks so much for all your inputs Adrian!

    One major thing I observed with the graph you provided is my JDBC node doesn't show the "Classpaths" tab; I've got all tabs showing except this.

  8. #8

    Default

    I figured it out Adrian!

    I will reach out when I face another hurdle... meanwhile, I shall work with our engineers.

    Happy Fri-yay!

  9. #9

    Default

    Hello Adrian,

    Can I please have the documents you shared with Sreekanth? I had been successful with connecting LAE to Hadoop previously. However, with the my OS upgraded to Windows 10, I lost this connectivity. Perhaps these documents may be able to assist with our troubleshooting/fixing the problem.

    Many thanks!

    Regards,

    tris

  10. #10
    Lavastorm Employee
    Join Date
    Nov 2012
    Location
    Warrington, UK
    Posts
    232

    Default

    The Hive ODBC Driver configuration guide is shipped with the LAE 6.1 installation software and should be in the following location (assuming the default installation directory):
    C:\Program Files\Lavastorm\LAE6.1\docs\lal

    It is also attached here for completeness:

    Lavastorm ODBC Driver for Hive Configuration Guide.pdf

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •